Search This Blog

Saturday, July 27, 2013

Arduino XYZ CNC / 3D Printer Do-over


Now that I've polished my Java skills over the past couple years I wanted to come back and improve a previous project of an XYZ CNC machine that was controlled by Arduinos and a crude PC Java program.

This is the hardware I'm using.  I've covered how to build all these pieces in earlier blog posts.  As I work through the pieces I'll refresh the info on the hardware and sketches.


I'll need to make some software updates to the Arduinos, but the original Arduino sketches are posted on my download pages.    I want to improve the handshaking, auto detect the arduinos, and process HPGL files better, plus make a better GUI.

There are three Arduinos.  The first is the XYZ motor controller that uses two motor shields from ADAfruit.  I posted a hack on how to attach two to the same Arduino.  I also replaced the motor driver chips with some higher capacity chips and put on heat sinks.  The reason one arduino does all this work is I absolutely want to synchronize the x, y and z moves.  I don't want to risk dropping a move or mistiming a move due to USB communication glitches.  This arduino gets HPGL format commands that give position from and to commands and it calculates all the sub stepping to get there.

The second arduino only controls the speed and direction of the cutter.  The cutter is a cordless drill motor and chuck controlled by some heavy duty MOSFET switches.

The third arduino is connected to position encoders and an LCD.  It reads back the position and also displays it and any other messages the machine has to say.   This is mostly used as a watchdog, since the steppers count position.  If the position gets off from what the machine thinks it should be I should stop and let the user figure things out.

Power is supplied by a old PC desktop power supply.

My PC original control program was very crude and didn't work if the Arduinos were in different COM ports and didn't have file spooling capability.  I'm way better at coding now, that was my very first program after "Hello World".

So my project this time is to write a respectable java program that will work for everyone, It will read/spool in a standard HPGL command file and control the CNC.  It will also have buttons for manual controlling the CNC.

I'm using Java and the netbeans development environment, which is free from oracle.
I'm starting with the basic Java serial tool I just wrote for the PC that will communicate with the Arduinos.

http://blog.workingsi.com/2013/01/revisiting-arduino-control-from-pc.html

The first piece will be a startup routine that finds all three Arduinos and establishes a link.  Each arduino sketch will be updated to send an initialization sequence I can look for.

After that there will be a choice of entering manual mode to push buttons on the GUI to move, or load an HPGL drawing file.  There should be a routine for setting the home position.

First I'm going to work on communicating with the position sensor I described in this post:
http://blog.workingsi.com/2011/02/position-sensor-for-arduino-xyz-cnc.html
I fired it up with the serial tool and the original code in the post is working fine, the encoder is constantly spitting out the position on the COM port as fast as it can.  For now I shouldn't need to alter the Arduino sketch.  My PC program will have to seek and identify this data stream and map it properly.

Next I fired up the XYZ stepper Arduino and that seems to be in good order as well.  The original sketch was built from this post.  You need to use the custom library to use all four steppers
http://blog.workingsi.com/2011/03/method-for-controlling-4-steppers-from.html
and the sketch for the XYZ control is posted on my download site:
http://code.google.com/p/arduino-java-xyzcnc/downloads/detail?name=XYZ_CNC_HPGL_r13.pde&can=2&q=#makechanges

Writing the Java to connect to all the Arduinos

First step is to perfect the serial port autodetection.  I'll use the touchport command and then look for a unique broadcast from each Arduino in order to assign them to ports.

After a long stall, I finally got this working in a recent project.
http://blog.workingsi.com/2013/07/lcd-system-monitor-for-pc-case-bling.html

I had been hung up from a mistake.  I used touchPort, to get the port names, and then instantiated a serial class to talk to each one.  However that process wiped out my list of ports so the loop failed.   Also I learned I have to wait and let the listener have time to hear each Arduino before moving on.  Here are the basic pieces.

This Java loop goes through all the ports:
     int iii=0;
        while (clientPortName.equals("")) {
                                                        
            //scan for devices and list
            serial.touchPort("", Integer.valueOf(serialRates.getSelectedItem().toString()));
                    jListPorts.setModel(new javax.swing.AbstractListModel() {
                    String[] strings = serial.portNames;
                    public int getSize() { return strings.length; }
                    public Object getElementAt(int i) { return strings[i]; }
            });
                            
            if (!serial.portNames[iii].equals("")) {        

                System.out.println("PORT:" + serial.portNames[iii]);
                setupCom(serial.portNames[iii]);
                // give it some time to hear the Arduino calling
                long currentTimeMillis = System.currentTimeMillis();
                while (System.currentTimeMillis()<currentTimeMillis+3000 ) {
                    //just hang out hoping for a letter
                }        
            }
          iii++;
          if (iii>19) {
              System.out.println("Exiting, Arduino not found");
              System.exit(1);
          }
        }



This routine below grabs each port and sets up a listener and sets a flag that breaks the loop above if the desired character is found.

       public void setupCom(final String portName) throws SerialException{
                //System.out.println(">>"+portName);
                serial.dispose();
                serial = new Serial(portName, Integer.valueOf(serialRates.getSelectedItem().toString()), 'E', 8, 2 );
                
        //Add the listener to pipe remote responses 
        serial.addSerialEventListener(new Listener_SerialEvent() {
            @Override
            public void SerialEventOccurred(int message) {
                String message_out = formatRead(message);           
                
                if (message_out.equalsIgnoreCase("Q")) {
                    System.out.println(portName + " detected Q - gotcha!!!");
                    clientPortName = portName;
                    send("Hello World!");  //respond so Arduino stops sending Q
                }
            }
        });
    }

I don't have to break the upper loop, I could go back and set up the port at the end of looping, but this is faster if I stop when the right port is found.

I've updated the serial Arduino tool with a field for the character is it going to look for to connect. In this picture the GUI has three Arduinos connected.  One is transmitting Z, one is busy, and one is transmitting A.  The one transmitting Z , COM4, is grabbed and now send and receive can begin.

Download the executable here:
https://code.google.com/p/arduino-java-xyzcnc/downloads/detail?name=SifishSerialArdunioTool.jar&can=2&q=

Now I will use multiple of these tools to search and connect to the three Arduinos.
I've got that code working, each Arduino is set to broadcast a different character to be found by the serial comm port scanner.  However that means I had to resurrect the old arduino code and modify it.

I had no trouble with the encoder program https://code.google.com/p/arduino-java-xyzcnc/downloads/list, the rotary encoder sketch.  I was able to add the establishContact() function from the Arduino serial response example.

The XYZ motor controller program is broken.  The old AFMotor.h libraries I had been using from Adafruit don't seem to be working, they don't compile in the new release.   I updated new library from here:
http://learn.adafruit.com/adafruit-motor-shield/downloads
that library seems OK, but my modified AFMotorA.h library for use in stacking shields is broken.  maybe I need to remake it.

The big problem turned out to be that now Arduino installs to the Program Files (x86) directory, which is read only and blocked by default.   I had to make all the library files unblocked by right clicking and selecting properties.  In the process I did remake the AFMotorA library.  All I did was search and replace AFMotor for AFMotorA and AF_Stepper with AF_StepperA, and microsteppercurve to microsteppercurveA in the .cpp, .h and keywords files.  Also rename the files AFMotorA.cpp and AFMotorA.h.   Now the old sketch compiles pointing to the AFMotor libraries.

I'm using three instances of my serial tool with only slight modification.  One for each arduino.
Had to do some mundane coding to pass the serial responses up to the main Arduino GUI by adding listeners to each of the serial port GUIs and passing them up to the main GUI.

The connection process takes so long that I want to move it into a thread by itself rather than locking up the GUI for 15 seconds.  that fixed a lot of weird problems I was having with the GUI not drawing, messages not scrolling etc.

Cool, now I have a main GUI that when you press connect, runs out and finds 3 different Arduinos and scrolls a log of what it is doing.   This simple task had eluded me for a while due to stupid mistakes.


Now the fun part, re-writing the code to actually move the XYZ machine.

Cleaning up the Arduino Encoder Program & Sketch

Updated the code to pull the position from the Arduino, parse the string and display it.  I'm finding the response time from the Arduino is really slow between when the encoders and LCD report the change, and when the string shows up on the serial bus.  I'm going to change the encoder code to only print to serial when there is a change to reduce the traffic and hopefully prevent the backlog.  It worked!  Now I only send serial traffic when there is a change, and response is much faster.

Realized I need a function for the Encoder program to tell it to reset it's counters.  That is for a Home position command.  Added that and updated the position encoder program.  Also found a bug that the position counters were unsigned integers!!  The LCD calculation seemed right but the serial information signs were mixed up.  Fixed it.  It is working much better now!

 /* Rotary encoder readout for XYZ CNC machine */

// Encoders connected as follows
// X encoder PIN A - Analog 0
// X encoder PIN B - Analog 1
// Y encoder PIN A - Analog 2
// Y encoder PIN B - Analog 3
// Z encoder PIN A - Analog 4
// Z encoder PIN B - Analog 5
// Pin C of all encoders is grounded, center pin of http://www.sparkfun.com/products/9117
// thus variable PINC will read this entire register simultaneously
// this avoids missing states of the encoder
//Based on 
//http://www.circuitsathome.com/mcu/reading-rotary-encoder-on-arduino
//Expanded to 3 encoders and written to be more clear and less over elegant

//Variables to set the counts/inch
#define TH_PER_IN  20 //lead screw threads per inch
#define CT_PER_REV  48 //encoder counts per revolution
// Thus the accuracy of the measurement 1 inch /(TH_PER_IN*CT_PER_REV)

//Variable to reverse the sign if the machine and encoder don't agree with your sense of things
// set to 1 or -1
#define REFLECTX -1
#define REFLECTY -1
#define REFLECTZ 1

#include <LiquidCrystal.h>
LiquidCrystal lcd(7, 6, 5, 4, 3, 2);

unsigned int olddata;
unsigned int newdata;
char newX;
char newY;
char newZ;
char oldX;
char oldY;
char oldZ;
long signed int counterX = 0;
long signed int counterY = 0;
long signed int counterZ = 0;
char inByte = 0;         // incoming serial byte
// table of meaning of states, input is two bits old value, two bits new value
// the encoder has either moved +1, -1 or stayed in the same place
int8_t enc_states[] = {0,-1,1,0,1,0,0,-1,-1,0,0,1,0,1,-1,0}; 

void setup()
{
  /* Setup encoder pins as inputs */
  // http://www.arduino.cc/en/Reference/PortManipulation
  DDRC = B00000000;   //Sets input direction all to analog inputs
  PORTC = B11111111;   //Sets pull ups on the analog pins to 1
  Serial.begin (9600);
  establishContact();
  Serial.println("Encoders Found");
  counterX = 0;
  counterY = 0;
  counterZ = 0;
  
  pinMode(13 ,OUTPUT);  //turn on backlight
  digitalWrite(13, HIGH); //turn on backlight
  lcd.begin(20,4);
  delay(500); //Optrex display needs to rest after a begin
  lcd.clear();
  lcd.setCursor(0,0);
  delay(100); //Optrex display needs to rest after setting cursor
  lcd.print("XYZ CNC Position");
  lcd.setCursor(0,1);
  lcd.print("X: ");
  lcd.setCursor(0,2);
  lcd.print("Y: ");
  lcd.setCursor(0,3);
  lcd.print("Z: ");
  
  //send initial position
   Serial.print(1000*counterX/(TH_PER_IN*CT_PER_REV*REFLECTX), DEC); Serial.print(":");
   Serial.print(1000*counterY/(TH_PER_IN*CT_PER_REV*REFLECTY), DEC); Serial.print(":");
   Serial.print(1000*counterZ/(TH_PER_IN*CT_PER_REV*REFLECTZ), DEC); 
   Serial.println(";");
      
}

void loop()
{
  //This loop needs to run fast to avoid missing encoder states
  newdata = PINC;
  
  if( newdata != olddata ) {
    
    newX = newdata & 0x03;  //mask out just the X bits
    oldX = olddata & 0x03;  //mask out just the X bits
    oldX = oldX << 2;  //move over to form address to table
    counterX += enc_states[oldX + newX] ;  //increment or decrement counter from table

    newY = newdata & 0x0C;   //mask out just the Y bits
    newY = newY >> 2;  //shift to LSBs
    oldY = olddata & 0x0C;   //mask out just the Y bits
    counterY += enc_states[oldY + newY] ; 
    
    newZ = newdata & 0x30;   //mask out just the Z bits
    newZ = newZ >> 4;  //shift to LSBs
    oldZ = olddata & 0x30;   //mask out just the Z bits
    oldZ = oldZ >> 2;  //shift to form address to table
    counterZ += enc_states[oldZ + newZ] ;
    
    // only send updates over serial bus when data has changed to reduce traffic
     Serial.print(1000*counterX/(TH_PER_IN*CT_PER_REV*REFLECTX), DEC); Serial.print(":");
     Serial.print(1000*counterY/(TH_PER_IN*CT_PER_REV*REFLECTY), DEC); Serial.print(":");
     Serial.print(1000*counterZ/(TH_PER_IN*CT_PER_REV*REFLECTZ), DEC); 
     Serial.println(";");
      
  }  else {
    // only update the display when the data is NOT changing, this speeds up polling
    lcd.setCursor(3,1);
    lcd.print(1000*counterX/(TH_PER_IN*CT_PER_REV*REFLECTX), DEC);
    lcd.print(" mils    ");
    lcd.setCursor(3,2);
    lcd.print(1000*counterY/(TH_PER_IN*CT_PER_REV*REFLECTY), DEC);
    lcd.print(" mils    ");
    lcd.setCursor(3,3);
    lcd.print(1000*counterZ/(TH_PER_IN*CT_PER_REV*REFLECTZ), DEC);
    lcd.print(" mils    ");
    
    // check if any serial messages have come in, since not busy
    if (Serial.available() > 0) {
      // get incoming byte:
      inByte = Serial.read();
      // reset position on indicated counter
      if (inByte == 'X')  counterX = 0;
      if (inByte == 'Y')  counterY = 0;
      if (inByte == 'Z')  counterZ = 0;
      if (inByte == '@' )  {
        counterX = 0;  counterY = 0;  counterZ = 0;
      }
      // any other serial traffic respond with current position
      Serial.print(1000*counterX/(TH_PER_IN*CT_PER_REV*REFLECTX), DEC); Serial.print(":");
      Serial.print(1000*counterY/(TH_PER_IN*CT_PER_REV*REFLECTY), DEC); Serial.print(":");
      Serial.print(1000*counterZ/(TH_PER_IN*CT_PER_REV*REFLECTZ), DEC); 
      Serial.println(";");
    }
    
  }
  olddata = newdata;
}

 void establishContact() {
  while (Serial.available() <= 0) {
    Serial.print('Q');   // send a capital Q to identify myself
    delay(300);
  }
}

I realized I need to be able to calibrate and send a coordinate to the encoders and have them update their math to scale that to the known zero position.  I'll do that once I make some progress on other fronts.

Writing the GUI to Manually Control The Motors

Set up a GUI to send manual commands to the CNC machine.  Functions for Jogging X,Y, and Z by a select-able step size.  Turn on and off the tool and change it's speed.  Display the encoder's value and the calculated location both.   Finally an interpolated move command to calculate the vector to an inputted coordinate.



Next I need to go back and clean up the programming for the heavy duty motor driver.
http://blog.workingsi.com/2011/05/notes-on-heavy-duty-motor-sheild-for.html
I used a bunch of mosfets and random bipolars to convert an Adafruit motor sheild to drive a drill motor.   I left this pretty rough.  I need to clean up the control sketch for the Arduino and test it out.   I set up a slider to control the speed.  I'm not sure if I'll do active motor braking or not, depends on how things work.

Decided to update the motor controllers and eliminate one of the Arduinos!

In the process of resurrecting the heavy duty motor shield circuit, and noticing that my blog post was kinda missing a lot of information, I discovered that Adafruit had updated their motor shield and no longer sold the old one I used.   They now support stacking, which I had hacked up before.
http://www.adafruit.com/products/1438
It's totally cheating to use their shield, and maybe pricey as well, but it will save me a lot of time and what I build will be repeatable by others.  Additionally it looks like I could stack and use a single Arduino to run all four motors X, Y, Z, and Tool.  They now have a dedicated PWM chip on board. Before as well the load on the Arduino PWM was too much for four motors and I backed off to two Arduinos.  I will still need to use the heavy duty Hbridge MOSFETs for the tool motor, but that is a minor mod.   I pulled the trigger to upgrade the motor shields to the new version.  It will save me an entire Arduino.  Too bad I already wrote code, but it will be worth it.

I'm rewriting the XYZ Arduino code now to support both the XYZ and the Tool speed and use the new Adafruit v2 motor shields stacked up.  Additionally I'm going to change the architecture of the system.  Previously I had the Arduino interpreting the HPGL code directly.  That was very limiting, but I did it because I didn't know Java at the time.   A better approach is to have the PC software do the hard work, and only send specific movement commands to the Arduino.  The Arduino will still linear interpolation to move the X and Y motors together to make smooth diagonals.  I don't think it's necessary to do linear interpolation with the Z axis.

New Arduino XYZTool code

Java program will need to send the following commands to the Arduino.
Semicolon indicates command termination and begin executing.
Command set proposed:

  • R; = Reset position counters to 0:0:0 (Set Home)
  • C; = Coordinates:  Report back the current stepper counter position over the serial link for use in calibrating and catching errors
  • Pn; = Z axis relative move n steps
  • Zn; = Move to absolute Z coord
  • Mx:y; = Move relative by x:y steps
  • Ax:y; = Move to absolute coordinate x:y
  • Tn; = Set tool speed to n.  May be positive or minus.  0 is off.
  • S; = Stop all motors
  • I; = power on/initialize motors

After each command the Arduino will send back characters:

  • B; - Arduino received the command and is busy executing it
  • R; - Ready for a new command
  • Cx:y:z - Arduino counters say this is the current coordinates

I added each of these commands to the buttons in the Java program.  Now I need to jump back to the Arduino to interpret them.

The Adafruit v2 motor shields came in the mail.   I assembled them exactly like the tutorial.  I stacked two shields, and shorted the address bit on the bottom one.  One for XY and one for Z and tool.   http://learn.adafruit.com/adafruit-motor-shield-v2-for-arduino/stacking-shields
I hooked up a tiny DC motor and three hobby steppers to the shields for a mock up to use while I write the code, without worrying about running the motors into a wall, etc.
A basic motor party script with both shields worked right off the bat.


Now I'm rewriting the motor control program for the Arduino to interpret the simple commands from the Java GUI, rather than try to decode HPGL all by itself.  That was limiting the functions I could perform.   It didn't take as long as I thought it would, because I was able to reuse the heavy lifting from the old program and just change the command interpretation loops.  I still need to add the timeout and error catching stuff but it's working pretty well already!

Arduino sketch for XYZ CNC control:
/*

Program to intepret CNC commands and activate motors
Uses Adafruit v2 motor shields http://learn.adafruit.com/adafruit-motor-shield-v2-for-arduino/overview

Commands that will be received:
R; = Reset position counters to 0:0:0 (Set Home)
C; = Coordinates:  Report back the current stepper counter position over the serial link for use in calibrating and catching errors
Pn; = Z axis relative move n steps
Zn; = Move to absolute Z coord
Mx:y; = Move relative by x:y steps
Ax:y; = Move to absolute coordinate x:y
Tn; = Set tool speed to n.  May be positive or minus.  0 is off.
S; = Stop all motors
I; = Power all motors/ initialize

After each command the Arduino will send back characters:
B; - Arduino received the command and is busy executing it
R; - Ready for a new command
Cx:y:z - Arduino counters say this is the current coordinates

 */

#define INCH 4000 //steps per inch of the machine = threads per inch * steps per rotation 
#define RPM 30 //speed of the stepper motors

#include <Wire.h>
#include <Adafruit_MotorShield.h> 
//#include "utility/Adafruit_PWMServoDriver.h"

// Create the motor shield object with the default I2C address for Z axis stepper and Tool
Adafruit_MotorShield AFMS_ZT = Adafruit_MotorShield(); 
// Or, create it with a different I2C address for stacking for X and Y axis steppers
Adafruit_MotorShield AFMS_XY = Adafruit_MotorShield(0x61); 

// Connect a stepper motor with 200 steps per revolution (1.8 degree)
// to motor port #2 (M3 and M4)
Adafruit_StepperMotor *stepperZ = AFMS_ZT.getStepper(200, 2);
// And connect a DC motor to port M1
Adafruit_DCMotor *myTool = AFMS_ZT.getMotor(1);
// Connect two stepper motor with 200 steps per revolution (1.8 degree)
// to motor ports on bottom shield
Adafruit_StepperMotor *stepperX = AFMS_XY.getStepper(200, 1);
Adafruit_StepperMotor *stepperY = AFMS_XY.getStepper(200, 2);

String inputString = "";         // a string to hold incoming data
boolean stringComplete = false;  // whether the string is complete
long previousMillis = 0;        //used for timeout counter
long timeout = 10000;           //timeout length

// Set up the basic machine position variables
signed long int Xpos = 0;
signed long int Ypos = 0;
signed long int Zpos = 0;

// Set up the destination machine position variables
signed long int newXpos = 0;
signed long int newYpos = 0;
signed long int newZpos = 0;

void setup()
{
  // start serial port at 9600 bps:
  Serial.begin(9600);
  inputString.reserve(200);
  establishContact();  // send a serial byte to establish contact until receiver responds 
  
  //Start the PWM
  AFMS_XY.begin();  // create with the default frequency 1.6KHz
  AFMS_ZT.begin();  // create with the default frequency 1.6KHz
  
  //initialize motors
  initMotors();
  
  // Home position
  resetPositionCounter();
  
  Serial.println("Stepper Controller Online;");
  reportCoords();
  
}

void loop()
{
  // if we get a valid byte, read analog ins:
  if (Serial.available() > 0) {
  
    // get the new byte:
    char inChar = (char)Serial.read(); 

    // if the incoming character is a newline, set a flag
    // so the main loop can do something about it:
    if (inChar == ';') {
      stringComplete = true;
    } else {
      // add it to the inputString:
      inputString += inChar;
    }
    
    // emergency stop command, process immediatly no matter what I'm doing
    if (inChar == '*') emergencyStop();
  
    if (stringComplete) {
      //Indicate busy 
      Serial.print("B;"); 
      previousMillis = millis();         //set the timeout counter to start

      //Echo the command
      //Serial.print(inputString);
  
      //decode and do what the command said
      switch( inputString[0] ) {
        case '*' : emergencyStop(); break;
        case 'I' : initMotors();  break;
        case 'T' : setToolSpeed();  break;
        case 'C' : reportCoords();  break;
        case 'P' : relativeZMove(); break;
        case 'Z' : absoluteZMove(); break;
        case 'M' : relativeXYMove(); break;
        case 'A' : absoluteXYMove(); break;
        case 'R' : resetPositionCounter(); break;
        default  : break;
      }
       
      //clear the command
      inputString = "";
      stringComplete = false;
  
      // Indicate ready
      Serial.print("R;"); 
      
    }  

  } 
}

//    else if ((previousMillis+timeout) > millis()) {
//       //Waited too long for the end of the command
//        intputString = "";
//        stringComplete = false;
//       Serial.println("E;");
//    } 

void establishContact() {   //for initial identification
  while (Serial.available() <= 0) {
    Serial.print("~");   // send a charachter to identify myself
    delay(300);
  }
}

void emergencyStop() {   //release all motors, limit hit or user intervention
      myTool->run(RELEASE);
      stepperX->release();
      stepperY->release();
      stepperZ->release();
      //Serial.println("All motors released;");
}

void setToolSpeed() {  
    signed long int newSpeed = extractCoord(inputString, 0);
    
    if (newSpeed>0) myTool->run(FORWARD);
    else myTool->run(BACKWARD);
    
    if (newSpeed == 0) myTool->run(RELEASE);
    else myTool->setSpeed(abs(newSpeed));
}


signed long int extractCoord(String numberString, int pos) {  //pulls the signed coord from the serial input string
  // pos indicates first or second coord desired
  
  int splitPos = 0;   //location of the : character
  signed long int coord = 0;   //temp storage for coords 
  signed int sign = 1;         //temp storage for coord sign 
  char c;  
  
  //First find the position of the :
    for (int i=1; i<numberString.length(); i++) {
      if( numberString[i] == ':' ) splitPos = i ; 
  }
    
  if (splitPos ==0)  {  // there is only one coord  
    for (int i=1; i<numberString.length(); i++) {
      c = numberString[i];
      if( c == '-' ) sign = -1; // capture the sign 
      if( c >= '0' && c <= '9') coord = (10 * coord) + (c - '0') ; // convert digits to a number     
    }
  } else {
  
    switch (pos) {
      case 0 :
      for (int i=1; i<splitPos; i++) {
          c = numberString[i];
          if( c == '-' ) sign = -1; // capture the sign 
          if( c >= '0' && c <= '9') coord = (10 * coord) + (c - '0') ; // convert digits to a number     
      }
      break;
      case 1: 
        for (int i=splitPos; i<numberString.length(); i++) {
          c = numberString[i];
          if( c == '-' ) sign = -1; // capture the sign 
          if( c >= '0' && c <= '9') coord = (10 * coord) + (c - '0') ; // convert digits to a number     
        }  
      break;
      default: //something is wrong, return 0;
      break;
    }
      
  }  
  return(coord * sign);
}
  
void resetPositionCounter() {
  Xpos = 0;
  Ypos = 0;
  Zpos = 0; 
  newXpos = 0;
  newYpos = 0;
  newZpos = 0;
}  

void reportCoords() {
    Serial.print(Xpos);Serial.print(":");Serial.print(Ypos); Serial.print(":");Serial.print(Zpos);Serial.println(";");  
}

void initMotors() {
  //initialize motors
  myTool->setSpeed(RPM);   
  myTool->run(RELEASE);
  stepperX->setSpeed(RPM);  
  stepperY->setSpeed(RPM);    
  stepperZ->setSpeed(RPM);    
  
  //step motors one step and back to power the coils and hold
  stepperX->step(1, FORWARD, SINGLE); 
  stepperY->step(1, FORWARD, SINGLE);
  stepperZ->step(1, FORWARD, SINGLE); 
  stepperX->step(1, BACKWARD, SINGLE); 
  stepperY->step(1, BACKWARD, SINGLE);
  stepperZ->step(1, BACKWARD, SINGLE); 
}

void relativeZMove(){
    zMove(Zpos + extractCoord(inputString, 0));
}

void absoluteZMove(){
    zMove(extractCoord(inputString, 0));
}

void relativeXYMove(){
    linearInterpolationMove(Xpos+extractCoord(inputString, 0), Ypos+extractCoord(inputString, 1));
}

void absoluteXYMove(){
    linearInterpolationMove(extractCoord(inputString, 0), extractCoord(inputString, 1));
}

void linearInterpolationMove ( signed long int newX, signed long int newY) {
  float distance = 0;
  int stepnum = 0;
  signed long int nextX;
  signed long int nextY ;
  signed long int oldX = Xpos;
  signed long int oldY = Ypos;

  //find the hypotenuse, the total distance to be traveled
  distance = sqrt((newX - oldX)*(newX - oldX) + (newY - oldY)*(newY - oldY) ) + 0.5;

  //round to integer number of steps that distance. Step by two to minimize 0 size steps.
  for (stepnum=0; stepnum <= distance; stepnum++) {
  
    //calculate the nearest integer points along the way 
    nextX = oldX + stepnum/distance*(newX-oldX);
    nextY = oldY + stepnum/distance*(newY-oldY);

    //move machine to that new coordinate, if 0 delta, don't move
    if ((distance < 4*INCH) && (distance > -4*INCH)) { //trap crazy values
      if ((nextX-Xpos) >=  1)  stepperX->step((nextX - Xpos), FORWARD, SINGLE); 
      if ((nextX-Xpos) <= -1)  stepperX->step((Xpos - nextX), BACKWARD, SINGLE);
      if ((nextY-Ypos) >=  1)  stepperY->step((nextY - Ypos), FORWARD, SINGLE); 
      if ((nextY-Ypos) <= -1)  stepperY->step((Ypos - nextY), BACKWARD, SINGLE);
    
    //update the machine current position
    Xpos = nextX;
    Ypos = nextY;
    } else {
      Serial.println("ERROR! Distance too big");
      break;
    }
  }
  
  // nudge to the exact final desired coord to correct for rounding down errors
  if ((abs(newX-Xpos)<6) && (abs(newY-Ypos)<6)) { //trap crazy values
      if ((newX-Xpos) >  0)  stepperX->step((newX - Xpos), FORWARD, SINGLE); 
      if ((newX-Xpos) < 0)  stepperX->step((Xpos - newX), BACKWARD, SINGLE);
      if ((newY-Ypos) >  0)  stepperY->step((newY - Ypos), FORWARD, SINGLE); 
      if ((newY-Ypos) < 0)  stepperY->step((Ypos - newY), BACKWARD, SINGLE);
  }
    //update the machine current position
    Xpos = newX;
    Ypos = newY;
}

void zMove (signed long int newZ) {
  //calculates steps to move, and moves, to new Z position requested
  signed long int distance = 0;
  signed long int oldZ = Zpos;

  distance = newZ - oldZ;
  
  if ((distance < 8000) && (distance > -8000)) {
    if (distance >=  1)  stepperZ->step(distance, FORWARD, SINGLE); 
    if (distance <= -1)  stepperZ->step(-1*distance, BACKWARD, SINGLE);
  } else {
    //movement requested is large and likely an error
    Serial.println("ERROR - Z axis range");
  }

  Zpos = newZ;  //update machine current position
}

Fired up the Java GUI and started clicking buttons.  It works!  All the buttons are doing the right things.  Immediately noticed I needed to make some GUI changes though.

-Stop button works fine, but once I've done a release I need to have a function for re powering the motors to recover.   Using the I; command for that;
-Tool speed scale is way too low, need to bump it up -
-Tool control, I didn't put on a direction button for forward/reverse - Done
-I need to look for and obey the busy flags from the arduino - Done, and made a indicator light

Things are working with the Java GUI, the busy light comes on, all the motors move the way I want.  I'm in business.   Here is what it looks like now.  It has gotten a bit busy.



Next step is to figure out what to do on the calibration page.
The way this should work is this:

  • Start with a calibrated, precisely measured square printed on paper.
  • Manually move the XY to a the lower left position
  • Hit a button to say it is there
  • Manually move the XY to the upper right position
  • Enter the size of the square and hit a button to say it is there
  • Lower the tool down to touch the work
  • Hit a button to say it is at 0
  • Lift it and put it down on an object of known thickness
  • Enter the height and hit a button to say it is there
  • Now the encoders and counters can calculate the size of their steps in inches/mm.
  • The units should be changed from counts to inches in the display
Here is the calibration GUI.  Added code to calculate the calibration and reset counters in the Arduinos as needed.

Next step is to go back and modify the GUI I previously made for Manual stepping to report values in the calibrated units that calibration measured or in raw steps as it does now.  Easy added the multiply to the calculated position display.

Alright... now off to figure out how to read and write graphics files.   Basic communication and control all seem to be working well with the mockup machine.


I posted the code for all the parts at
https://code.google.com/p/arduino-java-xyzcnc/downloads/list
This is a working version for manual movement of the machines, haven't implemented the file reading and transferring part yet.

'm rewriting my Java PC program.  The task remaining is to add the file reading and translation code to my communication and control software.  I need to be able to pick up some common format CNC files and execute them.

I want to leverage as much as I can of existing software, and just write a simple line by line translator to take an existing file format and convert it to the commands i need to send to the Arduino.

I used HPGL last time, but it is a dead language and is no longer supported by many tools and graphics editors.   It will be really hard to convert pictures into HPGL.  I need something new.

Researching 3D and CNC file formats... Again

Lots of random scribbling as I google....

http://answers.yahoo.com/question/index?qid=20110530081859AASSPLv
.stl – STL is a file format native to the stereolithography CAD software created by 3D Systems.
.iges – The Initial Graphics Exchange Specification (IGES) (pronounced eye-jess) defines a neutral data format that allows the digital exchange of information among Computer-aided design (CAD) systems.
.obj – OBJ (or .OBJ) is a geometry definition file format first developed by Wavefront Technologies for its Advanced Visualizer animation package. The file format is open and has been adopted by other 3D graphics application vendors. For the most part it is a universally accepted format.
.3ds – 3DS is one of the file formats used by the Autodesk 3ds Max 3D modeling, animation and rendering software.
Gerber - PC board drawing format

Wood carvers like STL files it seems
DXF shows up too - that is autocad

This site has a lot of STL files
http://www.cnc4free.org/

Tried to read one of the STL files.  It is not in ASCII and GIMP can't open it.  So that may not be the best option for me, to write an STL file interpreter.  Most people are trying to create these files, not read them.

b2G that will convert monochrome .bmp file to stl file

.bmp I have in the past written many programs that read and write .bmp files.  The format is pretty simple XY pixels and intensity.   I could use bmp as and easy option, but something more vectory seems better.

This guy is way ahead of me.   I'd have to switch to Linux to borrow his work.
http://www.securetech-ns.ca/camm-linux.html

G-code is used to run cnc machines at the lowest level.

Some tools for creating gcode http://replicat.org/generators

This software converts Eagle PCB into gcode
http://pcbgcode.org/index.php

cad.py looks like a convertor to gcode

I looked at gcode on wikipedia and it's fairly hairy.  Lots of commands to deal with.

Windows .bmp picture drawing

Starting to think I will write a simple bmp drawer to start with.
I want to get the tool running with something.

bmp files say their size and have rows and columns of pixels.  Easy to read.  I can use gimp to create files of any size, scale them, etc to make them suitable.  I'll start with black and white bmp files.

I need to do some googling to make sure there isn't some tool out there I can borrow.

If I choose to write it myself, the programming is going to go like this:
  1. Read in the bmp file into an array
  2. Find the outer extent on top, left and right
  3. move to the rough center
  4. put the pen down on a black dot.
  5. look for a neighbor that is black and move there
  6. continue until no neighbors are black
  7. lift
  8. find another pixel that is black
  9. move there
  10. repeat until the drawing is done
I have some old skill code that read bmp files, I will pull that out and use it to start.

Exploring other options

Found out vectorizing is called "autotracing".   That gave me another thing to google

There is an "autotracing" program on line. 
http://vectormagic.com/home
But the problem is I would then need to be able to read EPS, SVG or PDF files. Maybe the demo will be enough, but it looks like it costs money.

Found another that looks free and online.  Takes a graphics file and creates vector files.
http://www.autotracer.org/

  • Tried it, and EPS output looks like a usable, ASCII vector file.  
  • DXF file is ASCII readable as well.  I'm sure this could get hairy, Autocad is pretty complex.  But this file looks manageable.
  • SVG looks good too.  Seems to be a human readable graphics file that I could interpret
http://en.wikipedia.org/wiki/Scalable_Vector_Graphics
SVG could get out of hand, with lots of shapes and fonts to support.  However the basic file looks good
<path style="fill:#545a64; stroke:none;" d="M30 3L30 10C31.0595 7.46568 31.0595 5.53432 30 3z"/>


This means Move to 30:3, Line to 30:10, Bezier Curve to a list of coords, z=end.

Here is the spec on the SVG file format
http://www.w3.org/TR/2011/REC-SVG11-20110816/paths.html#PathDataMovetoCommands

Played with http://www.autotracer.org/ and I'm pretty happy with the files it produces.  It is really for 2D drawing but it makes simple vector files that are easy to turn into tool control files.  Gimp can read and display them.   Looks like a good quickie option to get off the ground.

This is a little graphic with some letters and a square:

<?xml version="1.0" standalone="yes"?>
<svg width="161" height="150">
<path style="fill:#ffffff; stroke:none;" d="M0 0L0 150L161 150L161 0L0 0z"/>
<path style="fill:#010101; stroke:none;" d="M1 1L1 109L117 109L117 1L1 1z"/>
<path style="fill:#ffffff; stroke:none;" d="M3 3L3 107L115 107L115 85C87.676 91.5053 87.5147 47.4563 115 54L115 3L3 3z"/>
<path style="fill:#010101; stroke:none;" d="M41 43L41 85L49 85L49 69C54.56 69 61.8856 70.2342 66.9568 67.5432C74.8268 63.3671 75.3453 49.6136 67.956 44.7423C61.5192 40.4989 48.4628 43 41 43M80 43L80 50L88 50L88 43L80 43z"/>
<path style="fill:#ffffff; stroke:none;" d="M49 50L49 62C66.8383 61.9882 66.8383 50.0118 49 50z"/>
<path style="fill:#010101; stroke:none;" d="M80 54L80 85L88 85L88 54L80 54z"/>
<path style="fill:#ffffff; stroke:none;" d="M107.043 59.9707C98.9687 61.792 101.299 81.8813 109.794 79.1489C117.684 76.611 116.364 57.8683 107.043 59.9707z"/>
</svg>

I will need to implement the Curve function in the Arduino, and convert the control letters to match just to make life easier.

Also found this code for running in a browser:
https://code.google.com/p/svg-edit/

It has a demo which is a fully functional svg editor that produces similar graphics files that can be edited.  Looks like a winner.
http://svg-edit.googlecode.com/svn/branches/2.6/editor/svg-editor.html


At this point it seems appropriate to start other posts with this topic, this post is getting too long and covers too much.

Wednesday, July 24, 2013

Arduino - avrdude: stk500_getsync(): not in sync: resp=0x00 error


This error happens whenever the Arduino IDE software can't talk to the Arduino board.
Code compiles and it gets to 90% on the progress bar and hangs.  Eventually it says
avrdude: stk500_getsync(): not in sync: resp=0x00

This could have any number of causes.  Just jotting down some of the things that have gone wrong and how I've fixed them.  Not detailed instructions, detail is everywhere on the web.


  • Is it plugged in?  Right port selected on Tools->Serial->Port.  Try a different cable.  Is the power light on the Arduino green?  Are the TX/RX lights flashing during upload?
  • Make sure the Arduino drivers are installed on your machine.  This usually happens when you load the software, but not always.
  • Go to the Computer->System->Device manager and find the arduinos.   Are they properly recognized?  if not, right click and update the driver with the Arduino drivers in the download
  • People often claim hitting the Arduino reset right after compilation will make it work.  That has never worked for me.
  • Sometimes the bootloaders get corrupted on the AVR chip.  To fix that you should get a AVR MKII ISP programmer from Digikey. http://www.digikey.com/product-search/en/programmers-development-systems/in-circuit-programmers-emulators-and-debuggers/2621880?k=avr%20mkii%20isp.   They are $37 but if you do enough Arduino work it is worth it.   There are ways to hack this up with another Arduino, but I've never tried it.   The Tools menu has settings for which programmer you have and to burn the bootloader.  You connect the programmer to the USB and plug it into the Arduino ISP header.  You have to supply 12V to the Arduino from the power jack.   Disconnect the Arduino USB connection and burn the bootloader.  Then upload your program with the programmer using the File->Upload using programmer.   Unplug the programmer from the Arduino and it should work normally.
  • Using the programmer you can just load the program through the ISP, without burning the bootloader, and sometimes everything will begin to work again.   Other times it will program fine, but the serial interface will still not work.  
  • I had two Arduino boards which none of the above worked.  The chip is probably dead..  I decided to try replacing the microcontrollers.   Order the ATMEGA328P-PU-ND http://www.digikey.com/product-search/en?WT.z_header=search_go&lang=en&site=us&keywords=ATMEGA328P-PU-ND&x=-1691&y=-51 for $3.16.   I pulled the old chip out of the socket and replaced it.  I burned the bootloader using the Arduino software.   Now it works!  Board is saved.

Friday, July 19, 2013

Nintendo 3DS power jack repair

My daughter's Nintendo 3DS wouldn't charge well any more.  The power plug on the 3DS was obviously very worn out and bent looking, and even the charging base wasn't working very well.  The charging light would flicker yellow and lots of fiddling might make it charge if you were lucky.

It was clear that there was a broken or barely hanging on electrical connection in the power jack.

I put this off a while, thinking it would be hard, but it turned out to be a really easy repair.

You can buy replacement jacks on ebay and amazon for <$7 including shipping.  Make sure to buy it somewhere from a US vendor, not from a vendor shipping from China/Honk Kong.  You would still get it, but it will take you weeks.

Found them on amazon.
Power Jack Socket Dock Connector Charger Charging Port Nintendo 3DS Replacement

Bought this replacement jack off ebay for $6.35, free shipping.  Came in two days!
http://cgi.ebay.com/ws/eBayISAPI.dll?ViewItem&item=330957509059

Power Jack Socket Dock Charger Charging Port Replacement for Nintendo 3DS N3DS

Watched this video.  Not bad this kid shows you how to get the cover off.
how to fix nintendo 3ds charging port - YouTube

You need a really small Phillips head and optionally a plastic pry tool.  If you are careful or use your fingernail you don't need the pry tool.  You can buy them on eBay for pennies.

After the 4 screws on the battery cover, There are nine screws and one on the game slot you have to take out.   They are very small and were in really tight.  A good screwdriver is needed, and press hard.



Take out the SD card and battery and any games and the bottom half of the DS pulls right off.



Watch out for the flex cables, just leave them connected and don't pull them.

I found the power jack actually just sitting in place.  All of the solder connections were broken loose and I could just pull it off.   The new connector went right on top of where the old one was.






I used a soldering iron and solder to attach the 6 tabs to the pads.  It's obvious how it goes.  My solder job was crap because it's small and my iron was big, however it was still no problem.

Put the lid back on, screws back in, battery, SD card, cover.  Powered up.  Ba da bing.  Plugged in the charger.  Yellow light.  Win!

This is a really easy repair.  If the connector had not been obviously broken, I would have had to use the soldering iron and the little vacuum desoldering tool to remove it.   Available at radio shack.  Luckily I didn't have to do any desoldering.


Monday, July 15, 2013

Spoofing OBDII to pass emissions test

This is a bad idea, but I just wanted to get it out of my system.





Today I had to take my old Honda for a state emissions test.   The test consists of looking under your car with a mirror, and plugging in the OBDII for about 2 minutes.  They may have some sensors for the exhaust gases, but it's not obvious.   It's my understanding that they poll the OBDII for any error codes, and that is just about it.   I passed, but had to wait in line long enough to make me think subversive thoughts.

I was reminded of the work I did years ago building an OBDII reader.  But how hard would it be to make a spoofed OBDII device?   It seems like it would be pretty easy to disconnect the OBDII and put a micro-controller behind the OBDII connector to read back a clean bill of health.

The usual disclaimer, I would never do such a thing and I don't advocate any one else do it.  But as an intellectual exercise I thought I'd look into the feasibility.   I wonder if there are protections against this sort of thing?

There are no new ideas under the sun, so a little googling found some people thinking of the same thing:

http://forums.nasioc.com/forums/showthread.php?t=753331
http://grassrootsmotorsports.com/forum/grm/learn-me-obd-ii-emulation/58941/page1/
http://my.is/forums/f114/passing-obd2-emissions-standalone-339716/index5.html
http://forum.miata.net/vb/showthread.php?t=143853

Ease Diagnostics makes an OBD-II emulator - 
http://www.obd2.com/emissions/data/emissions_ver_test.htm
Used for testing the emissions testers.  They have this little box telling you why you can't use if for spoofing:

SECURITY FEATURES
To prevent the EASE OBD II Verification Tester from being substituted as a "clean" vehicle in an emissions inspection, we have built in the following security features.  
  • The controller ID is permanently set to $FD.  This value is not an acceptable controller ID.
  • The number of PIDs is always 32.  On an actual vehicle this would mean all the PIDS were supported.  This is not likely to happen. 
  • The AC Sys Refrig I/M Monitor is always Supported and Not Complete. No vehicle currently supports this I/M monitor. 

Looks like the OBDII has some vehicle information, rpm, etc.  All which would be easy to duplicate.  So the first step would be to read your present OBDII, clone the information, and write it back to the spoofer.

One of my raspberry PI's might be best, because it has lost of memory/disk for stored values and GPIO to write a bit banged serial interface, and you can connect a keyboard, display and mouse for the programming phase.   An arduino might be able to pull it off, but I suspect it would require two programs.  One to read and one to spoof.

This page has some info on what the emissions test is looking for
http://www.scangauge.com/tips-and-tricks/finding-incomplete-obdii-tests-using-the-scangauge-ii/

I'll put this idea on the shelf for now and see if I decide to build it later.

Sunday, July 14, 2013

LCD system monitor for PC case bling using Arduino

This post is about a project to build a CPU/system monitor display for the front of a Windows desktop computer. Lights that get brighter as the CPU load increases.  A little fun and bling for the new machine.



I'm building a new high end desktop PC, and due to a delay in getting the processor shipped to me I've been wasting time thinking of what accessories to fill up those unused 5.25" bays.

I saw some widgets with LCDs to display temperature and fan speeds, etc.  However after reading reviews I'm not thrilled with any of them.   Additionally they don't read the real temperatures, they have probes you glue on.




That got me thinking of a quickie project that would be fun.   I'll make a panel LCD myself,   I'd like to display the CPU load, or other parameters, and temperatures if I can.  I know it's fairly hard to pull the temperatures from motherboards due to the wide variety of designs.

The basic design will be this.  I'll use an Arduino with an LCD plugged into either a USB port or onto a USB header on the mother board.   It will be a fairly simple serial client that displays whatever text you send it.
Running on the PC will be a Java program feeding info to the USB serial com port.   The Java program will pull the CPU load and System temperatures and format it for display.   I could do color changes, whatever to make it fancy.

The first task is to access the desired information from the Java program running on the Windows PC.

Some useful posts

  • http://stackoverflow.com/questions/47177/how-to-monitor-the-computers-cpu-memory-and-disk-usage-in-java
  • http://stackoverflow.com/questions/634580/cpu-load-from-java
  • http://sellmic.com/blog/2011/07/21/hidden-java-7-features-cpu-load-monitoring/


Some info can be pulled from within Java, but it is fairly limited.  I tried this code pasted into a little netbeans program

OperatingSystemMXBean OS = ManagementFactory.getOperatingSystemMXBean();    
        jLabelCPUs.setText("Number of CPUs: "+ Integer.toString(OS.getAvailableProcessors())+ " " +OS.getArch());;
        jLabelLoad.setText("CPU load: "+ Double.toString(OS.getSystemLoadAverage()));
     
The OS.getSystemLoadAverage() always Returns -1.   A little more reading reveals that this doesn't work on Windows.  Not definitive but discouraging.   The CPU architecture commands do work though.

Instead I thought I'd run a windows command line program from within Java and pull the output.  This approach worked much better.

http://stackoverflow.com/questions/9097067/get-cpu-usage-from-windows-command-prompt

There are two basic approaches.  wmic and typeperf.  Entered these commands into a cmd window and got back a string of the CPU load.   Both have some pretty obscure command arguments.
        wmic cpu get loadpercentage
        typeperf -qx "\Process" > config.txt typeperf -cf config.txt -o perf.csv -f CSV -y -si 10 -sc 60

Now to embed this in Java and get the output back to the Java program.
This Java code runs the cmd line wmic.

try {
            Process p = Runtime.getRuntime().exec("wmic cpu get loadpercentage");
} catch (IOException ex) {
            System.out.println("cmd line failed");
            Logger.getLogger(BlingGui.class.getName()).log(Level.SEVERE, null, ex);
}

Looking for how to get the output back, I see I just gotta use buffered stream to read it:

Back to polishing up the windows cmd line commands to get the data I want.

Wmic is awesome!   I can get more than cpu load.  All sorts of process and machine info.
wmic cpu get currentClockSpeed
wmic cpu get loadpercentage

Typeperf is also very useful.
http://technet.microsoft.com/en-us/library/bb490960.aspx

This command gives processor and memory usage in running 1 second intervals.  Awesome.
typeperf "\Memory\Available bytes" "\processor(_total)\% processor time"


This makes just one sample, also seems a little slow to respond
typeperf -sc 1 "\processor(_total)\% processor time"
typeperf command format is a mess to escape in Java, but got it to work like this:
Process p = Runtime.getRuntime().exec("typeperf \"\\processor(_total)\\% processor time\" ");
This makes just one sample at a time.
typeperf -sc 1 "\processor(_total)\% processor time"

Here is the output of typeperf writing to a hack GUI in Java. The yellow is the CPU%.  Obviously haven't done the formatting yet.   Making progress!



I wasted some more time looking for how to get the temperature command line, and it is not easy. I'm avoiding using somebody else's freeware program like CPUID or speedfan. Both of which work but won't get me to my goal. I'm going to stick with CPU load, memory and frequency data for the prototype.

A little more work and I have a Java performance meter program. Grabbed the cmd line output, did some string manipulation and formatting.  I haven't added the serial client stuff yet to talk to the Arduino yet.  This demos all three methods.  Java commands, wmic and tyeperf commands.  The typeperf is the one running the progress bar.



The executable is here: https://code.google.com/p/arduino-java-xyzcnc/downloads/list
This is the gist of the code, minus the GUI stuff.

 public BlingGui() {
        initComponents();
        this.setVisible(true);
        OperatingSystemMXBean OS = ManagementFactory.getOperatingSystemMXBean();
        
        jLabelCPUs.setText("Number of CPUs: "+ Integer.toString(OS.getAvailableProcessors())+ " " +OS.getArch());;

      
        Timer t_update = new Timer(10000, new ActionListener(){
            private String t1;
            @Override
            public void actionPerformed(ActionEvent e){
                try {
                    Process w1 = Runtime.getRuntime().exec("wmic cpu get CurrentClockSpeed");
                    BufferedReader stdInput1 = new BufferedReader(new InputStreamReader(w1.getInputStream()));
                    BufferedReader stdError1 = new BufferedReader(new InputStreamReader(w1.getErrorStream()));
                    
                    while ((t1 = stdInput1.readLine()) != null) {
                        if(t1.length()!=0)
                            if( Character.isDigit( t1.charAt( 0 ) )) jLabelFreq.setText("Clock: "+ t1.trim() +" MHz");
                    }
                    

                } catch (IOException ex) {
                    System.out.println("cmd line failed");
                    Logger.getLogger(BlingGui.class.getName()).log(Level.SEVERE, null, ex);
                }
                    
                }
            });
            t_update.start();
        
        
        try {
            
            // execute windows command line:  typeperf -sc 1 "\processor(_total)\% processor time"     
            Process p = Runtime.getRuntime().exec("typeperf \"\\processor(_total)\\% processor time\" ");
            
            BufferedReader stdInput = new BufferedReader(new InputStreamReader(p.getInputStream()));
            BufferedReader stdError = new BufferedReader(new InputStreamReader(p.getErrorStream()));


            // read the output from the command
            System.out.println("Here is the standard output of the command:\n");
            String s;
            while ((s = stdInput.readLine()) != null) {
              s = s.replace("\"", "");
              String[] sArray = s.split(",");
              s = sArray[sArray.length-1].trim();
              jLabelLoad.setText("CPU Load: " + s + "%");
              
             try {
                jProgressBarCPU.setValue( Integer.valueOf(s.indexOf('.') ));
                double dd = Double.valueOf(s);
                int gg = (int) dd;
                jProgressBarCPU.setValue( gg);
                if (gg<33) jProgressBarCPU.setForeground(Color.green);
                else if (gg<66) jProgressBarCPU.setForeground(Color.orange);
                else jProgressBarCPU.setForeground(Color.red);
             } catch (Exception e) {
                //Sometimes this is not a number
               
             }
              
            }  
        
            // read any errors from the attempted command
            System.out.println("Here is the standard error of the command (if any):\n");
            while ((s = stdError.readLine()) != null) {
                System.out.println(s);
        }
                    
            
        } catch (IOException ex) {
            System.out.println("cmd line failed");
            Logger.getLogger(BlingGui.class.getName()).log(Level.SEVERE, null, ex);
        }    
        
    }
    

Next step is to combine this with my Arduino serial communication program to send the CPU load data to an external Arduino with LCD shown in this post:
http://blog.workingsi.com/2013/01/revisiting-arduino-control-from-pc.html
This is just for prototyping, I'll build a custom LCD unit that fits in a drive bay once I get the s/w working.

For proof of concept I did a Frankenstein and simply pasted the CPU monitor GUI class right into my serial comm tool.   That way I use that tool to find the Arduino comm port and connect, and then the CPU monitor   kicks in and sends the CPU load string information as if I were pushing the send button on the GUI.

The Arduino is just running the Examples->Liquid Crystal->SerialDisplay sketch right out of the box.
The hardware is an Ardino with a large 4x20 LCD connected just like the Arduino example says.   I won't repeat all that here.  I reused this from http://blog.workingsi.com/2011/02/arduino-lcd-countdown-clock.html.   Everybody should have an Arduino with an LCD on it lying around for emergencies.

W00t!  It works!.   Here is the PC running the serial comm tool, with the new CPU GUI below it.  On top of the screen is the Arduino/LCD displaying the CPU load every second.


Now to clean up the code and make the LCD display a bar graph instead of just number.

The Arduino code need some upgrades.
  • Sleep whenever it hasn't gotten data in a while.  Turn off the LCD backlight and blank the display.
  • Initially send a character like "AAAAA" until it's connected so the Java GUI can find it and initialize communication
  • Eventual features using the Analog pins to control some LEDs brightness or color
Here is the Arduino sketch.  I'd like to keep it simple like this.  It can also display messages as I choose if I leave the brains in the Java host side:

/*
  LiquidCrystal Library - Serial Input

This sketch displays text sent over the serial port
 (e.g. from the Serial Monitor) on an attached LCD.

 The circuit:
 * LCD RS pin to digital pin 7
 * LCD Enable pin to digital pin 6
 * LCD D4 pin to digital pin 5
 * LCD D5 pin to digital pin 4
 * LCD D6 pin to digital pin 3
 * LCD D7 pin to digital pin 2
 * LCD R/W pin to ground
 * 10K resistor:
 * ends to +5V and ground
 * wiper to LCD VO pin (pin 3)

  
 http://arduino.cc/en/Tutorial/LiquidCrystalSerial
 */

// include the library code:
#include <LiquidCrystal.h>
const int backlight = 8;   // the LED backlight is wired to pin 8

long previousMillis = 0;            //stores the last time the display got data

// initialize the library with the numbers of the interface pins
LiquidCrystal lcd(7, 6, 5, 4, 3, 2);

void setup(){
    // set up the LCD's number of columns and rows:
  lcd.begin(16, 2);
  // set up the LCD backlight
  pinMode(backlight, OUTPUT);
  digitalWrite(backlight, LOW);
  // initialize the serial communications:
  Serial.begin(9600);
  establishContact();  // send a byte to establish contact until receiver responds
}

void loop()
{
  // when characters arrive over the serial port...
  if (Serial.available()) {
    // wait a bit for the entire message to arrive
    delay(100);
    // clear the screen
    lcd.clear();
    // wake up and turn on the backlight
    digitalWrite(backlight, HIGH);
    previousMillis = millis();
    // read all the available characters
    while (Serial.available() > 0) {
      // display each character to the LCD
      lcd.write(Serial.read());
    }
  } else if (millis() - previousMillis > 10000) {
     //turn off the backlight and clear the display
     digitalWrite(backlight, LOW);
     lcd.clear();

  }
}

void establishContact() {
  while (Serial.available() <= 0) {
    Serial.println("Z");   // send an initial string
    delay(300);
  }

}

Now the display wakes up when it gets a message, and goes blank 10 seconds after the last update.  Perfect.

None of the LCD's in my drawer will fit behind a 5.25" drive bay panel.   Actually some of the tiny ones will do but I want something snazzy.
The panel on the PC is about 1 5/8 * 5 3/4, (42mm x146mm)  and the LCD will need to be smaller.   I probably will need to use a 2x20 to get the right shape, the 4x20s are too tall.
It is also clear the Arduino itself is too tall and I will have to use a right angle header to connect to the LCD.

It occurred to me that I would also like to have some connector/header on the front I can use to access a few of the pins of the Arduino so I can use it for development and whatever GPIO project I dream up.

This vacuum fluorescent one looks cool, a bit pricey at $29
20x2 Character VFD (Vacuum Fluorescent Display) - SPI interface
http://www.adafruit.com/products/347
http://www.adafruit.com/datasheets/20T202DA2JA.pdf
It also has a serial interface.  That will save pins but require the code to be changed.
Outer dimensions are 37mm x  116mm.  It will just barely fit.


http://www.adafruit.com/products/399#Downloads
This one is 36x80mm and has a three color backlight.  $14. Only 2x16 chars
RGB backlight negative LCD 16x2 + extras

Looking at digikey...

http://www.digikey.com/product-detail/en/M0220SD-202SDAR1-1G/M0220SD-202SDAR1-1G-ND/1701410
MODULE VF CHAR 2X20 5.34MM | M0220SD-202SDAR1-1G | M0220SD-202SDAR1-1G-ND | Digi-Key Corp.
$31, 116mm x 37mm, 20x2, parallel interface like I'm used to.  However price is a bit steep, not sure if VF is worth it.

Searched a while, decided the VF displays are too expensive and I like the multicolor backlight price and convenience of LCD.

Digikey has this one, looks similar to the one on Adafruit.
http://www.digikey.com/product-detail/en/NHD-0216K1Z-NS(RGB)-FBW-REV1/NHD-0216K1Z-NS(RGB)-FBW-REV1-ND/2172436
LCD MOD CHAR 16X2 RGB TRANSM | NHD-0216K1Z-NS(RGB)-FBW-REV1 | NHD-0216K1Z-NS(RGB)-FBW-REV1-ND | Digi-Key Corp.

I decided to get the Adafruit display, they say they have resistors onboard for the backlight, and it comes with a trim pot.

The display came.  I'm going to mount it in an old 5.25 bay blank plastic cover from an old PC.  Here is the display sitting on top of the plastic blank bay cover.


My plan is to wire it up just like the Adafruit tutorial
http://learn.adafruit.com/character-lcds/rgb-backlit-lcds
And to add a small header connector to some unused pins to come out the front of the PC for development projects.

I cut a hole in the plastic panel to fit the LCD with a Dremel. It didn't come out as well as I'd hoped.  I'm going to have to fill and paint or try again with another blank.   Forging forward to work out the rest of the details before wasting time remaking something that may have to change anyway.

I'm using the header pin strips that came with the LCD to connect some scrounged IDE ribbon cables to the display.  I had thought I'd connect that directly to the Arduino with another header, but the power, ground and potentiometer connections will be awkward and messy.   Plus I wanted to put the analog pins on another header that sticks out the panel so that I can use them for future prototyping projects.    So I decided to bite the bullet and use an $6 adafruit prototyping shield to serve as a patch board and the $2 headers.  These I had in my parts bin already from past projects.
Adafruit Proto Shield for Arduino PCBShield stacking headers for Arduino (R3 Compatible)
http://www.adafruit.com/products/55
http://www.adafruit.com/products/85

The IDE ribbon cables are 17 pins wide, and I need 18, so I'll take advantage of the fact that the R/W pin is grounded and just short them together on the LCD.

Here is the protoboard soldered up with the headers and pot, connecting the cable header in the middle to the Arduino headers.   Careful consideration of the direction and the connections were all pretty easy to make.

Plugged in the IDE ribbon cable to the headers.   I used an ohm meter to double check the wiring.  I found no mistakes.  First time that ever happened.


Attached the Arduino to the protoboard, plugged in the USB.  I copied the code directly from the Adafruit page since the hardware was exactly the same as they show.  http://learn.adafruit.com/character-lcds/rgb-backlit-lcds.  Plugged in the USB, uploaded the sketch, twisted the pot until the contrast showed the words.  Win on the first try!


Rather than remake the front panel, I put some electrical tape on the edges of the display to stop the light leakage, and used foam tape to mount the display in the panel blank.   I decided to forge ahead and throw it together to see if there were any issues and polish up the software.  It does look a tad ratty.  LCD is crooked and the cutout is ragged.  The picture even makes it look worse than it does in real life.

Hopefully I'll go back and remake the front panel and mount the LCD better later.


Next step is to update the previous serial display sketch to handle the multicolor backlight.  Then I need to polish up the PC Java program to send it the CPU load.

I combined the multicolor LCD Arduino code with the earlier serial LCD program and it's working great.  I write to the display, it pops up blue, then goes to sleep after 10 seconds.   I won't post the code yet because there are two issues.  
1) I need someway to control the multiple colors
2) The serial program only writes to the first line of the display.  It will wrap to the second line in a buggy way and skip many characters

I need to add some control characters to set the colors and which line I want the text to show up on.

This code did the trick!  Intercept the characters and test them.  Now I can change the color and move the cursor with the special characters.  I want to use some more obscure characters but this was a first pass.  

    // read all the available characters
    while (Serial.available() > 0) {
      // display each character to the LCD
      //lcd.write(Serial.read());
      
      char temp;
      temp = Serial.read();
      if (temp=='^') setBacklight(0, 255 , 0);
      else if (temp=='<') setBacklight(255 , 0, 0);
      else if (temp=='>') setBacklight(0, 0 , 255);
      else if (temp=='@') lcd.setCursor(0,0);
      else if (temp=='$') lcd.setCursor(0,1);
      else lcd.write(temp);
      

    }

This chart shoes how to combine the RGB to get different colors
http://dba.med.sc.edu/price/irf/Adobe_tg/models/rgbcmy.html



I also found that many of the combinations, at reduced brightness have a bad flicker.  That is probably due to the PWM on the arduino being too low.   Since I don't want to mess up the delay timers, I'm just going to pop the brightness on those colors.   


Now my if statement is getting hairy, but I have a good collection of color:

      char temp = Serial.read();
      if (temp=='^') setBacklight(0, 255 , 0);        //green
      else if (temp=='<') setBacklight(255 , 0, 0);   //red
      else if (temp=='>') setBacklight(0, 0 , 255);   //blue
      else if (temp=='{') setBacklight(0, 255 , 128); //cyan
      else if (temp=='}') {
        brightness = 255;  //must max out to prevent flicker
        setBacklight(255, 100 , 0);                    //orange 
      }
      else if (temp=='|') {
        brightness = 255;  //must max out to prevent flicker
        setBacklight(128, 0 , 255);                    //purple  
      }
      else if (temp=='~') {
        brightness = 255;  //must max out to prevent flicker
        setBacklight(255, 0 , 255);                    //magenta 
      }
      else if (temp=='&') {
        brightness = 255;  //must max out to prevent flicker
        setBacklight(255, 255 , 255);                   //grey
      }
      else if (temp=='_') {
        brightness = 255;  //must max out to prevent flicker
        setBacklight(255, 255 , 0);                   //yellow
      }
      else if (temp=='@') lcd.setCursor(0,0);
      else if (temp=='$') lcd.setCursor(0,1);
      else lcd.write(temp);
      
I think the Arduino code is done, I posted the whole thing here
http://code.google.com/p/arduino-java-xyzcnc/downloads/detail?name=multicolor_lcd_serial_display.ino
It is basically the Adafruit example plus the serialdisplay example and the color selection code.

Now to switch back to the Java PC side to clean up the code sending information.
I need to format the information, include the control characters.  Also I've noticed I need to kill the typeperf process when I exit, I've been getting zombies.

Fixed the zombies by putting a custom window listener on the window exit command, that does a p.stop() on the typeperf runtime.   

Updated my serial arduino tool to automatically touch each port and read from it.   Once the desired character stream, in this case QQQQQ..... appears the program grabs that comm port and begins sending the CPU information.

Spending a little more time on typeperf to try to get it to give me the processor load, processor clock speed and memory all in one command.  wmic is good at processor clock, but I have to spawn multiple processes to get both typeperf and wmic running at once.  Typeperf does the 1 second sampels all by itself and sends the data back on one line.

typeperf -qx dumps all the possible information.
some juicy lines....I marked the ones that I liked in red.  Some didn't seem to work and gave back 0.

\Processor Information(0,_Total)\% of Maximum Frequency
\Processor Information(0,1)\% of Maximum Frequency
\Processor Information(0,0)\% of Maximum Frequency
\Processor Information(_Total)\Processor Frequency
\Processor Information(0,_Total)\Processor Frequency
\Processor Information(0,1)\Processor Frequency
\Processor Information(0,0)\Processor Frequency
\Processor(0)\% Processor Time
\Processor(1)\% Processor Time
\Processor(_Total)\% Processor Time
\Memory\Page Faults/sec
\Memory\% Committed Bytes In Use
\Memory\Available KBytes
\Memory\Available MBytes
\PhysicalDisk(0 C: D:)\Disk Transfers/sec
\PhysicalDisk(_Total)\Disk Transfers/sec
\PhysicalDisk(0 C: D:)\Disk Reads/sec
\PhysicalDisk(_Total)\Disk Reads/sec
\PhysicalDisk(0 C: D:)\Disk Writes/sec
\PhysicalDisk(_Total)\Disk Writes/sec
\PhysicalDisk(0 C: D:)\Disk Bytes/sec
\PhysicalDisk(_Total)\Disk Bytes/sec

to use any of these just put them in quotes after typeperftypeperf "\PhysicalDisk(_Total)\Disk Bytes/sec"
To dump all the ones I want, I used this command:

typeperf "\Processor Information(0,0)\Processor Frequency" "\Processor(_Total)\% Processor Time" "\Memory\% Committed Bytes In Use" "\PhysicalDisk(_Total)\Disk Bytes/sec"

Funny, using this command I discovered on my work laptop that the power management settings were running the CPU capped at 800MHz.  Turning up the slider to "performance" got the CPU speed up to 2534.  It is much peppier now.  A nice Easter egg.

Here is the typeperf command in Java with all the escapes
final Process p = Runtime.getRuntime().exec("typeperf "
                    + "\"\\Processor Information(0,0)\\Processor Frequency\" "
                    + "\"\\processor(_total)\\% processor time\" "
                    + "\"\\Memory\\% Committed Bytes In Use\" "
                    + "\"\\PhysicalDisk(_Total)\\Disk Bytes/sec\" "
                    + "");

I wrote some code to slice up the response and pull out the numbers, and change color depending on load.   Ha ha funny part is the new machine is so bloody fast it is almost impossible to max out the CPU.  I finally had to download Sisoftware sandra benchmark tool.  No amount of running youtube videos even got me to 10% CPU load.

Here is a snippet of the Java that decodes the typeperf response, makes a bar graph and changes the color of the display using the control characters.

            String sp;
            while ((sp = stdInputp.readLine()) != null) {
                //Parse and clean up the typeperf response
                jMessageArea.setText(sp);
                sp = sp.replace("\"", "");                
                String[] spArray = sp.split("\\s*,\\s*");
                try {
                    //System.out.println(spArray[1]+" "+ spArray[2]+" "+ spArray[3]+" "+ spArray[4]);                                     
                    
                    float CPUfreq =  Float.valueOf(spArray[1]);
                    float CPUload = Float.valueOf(spArray[2]);
                    float MemUsed = Float.valueOf(spArray[3]);
                    float diskBytes = Float.valueOf(spArray[4]);
                    
                    int CPUf = (int) CPUfreq;
                    int CPUl = (int) CPUload;
                    int memu = (int) MemUsed;
                    
                    int bargraph = (int) (CPUload*16/100 + 0.5);
                    String bars = "";
                    for (int jj=0; jj<bargraph; jj++) {
                        bars=bars.concat("=");
                    }
                    
                    String displayString = bars + "$" + Integer.toString(CPUf) + "MHz  Mem" + Integer.toString(memu)+"%";

                    // add color control characters based on load
                    if (CPUload>79) displayString = "<" + displayString;       //red  
                    else if (CPUload>49) displayString = "}" + displayString;  //orange
                    else if (CPUload>24) displayString = "_" + displayString;  //yellow
                    else if (CPUload>14) displayString = "^" + displayString; //green
                    else displayString = ">" + displayString;  //blue
                    
                    send(displayString);
                                                                               
                }
                catch (Exception e) {  
                    System.out.println("Exception "+ e.toString());
                }
            }

I left the Java GUI fairly rough, since it will run in hidden mode most of the time.  The Hide button sets the form to run invisibly.   This is detecting several Arduinos on the system at once, and automatically picking the one that is sending the QQQQQ stream identifying itself.



I need to put the .jar file somewhere it will start when the computer starts, so I put the .jar executable into the startup folder.
http://windows.microsoft.com/en-us/windows-vista/run-a-program-automatically-when-windows-starts

Here are a couple shots and a movie of the display responding and changing color.   Note the processor clock jumps up and the bar graph across the top moves to indicate greater load.  You can see in detail the crap job I did cutting the hole in the panel.




The video is a bit like a minute of watching paint dry, but you can see the processor benchmark test finish and ramp down the processor load.  In these videos I wasn't showing the Memory load yet.





The java executable is here to download:
http://code.google.com/p/arduino-java-xyzcnc/downloads/detail?name=SifishCPUMonitor.jar
If you try to run this, the usual caution that you need Java JRE installed on your machine, plus you have to copy the rxtx jar and dll to the proper places from the Arduino directory.  The java program will prompt you to do it if you don't do it right.  It's covered in my other posts.

The source and Arduino code is also on my download page
https://code.google.com/p/arduino-java-xyzcnc/downloads/list


Win!  A quick project and my hot new PC has something nobody else's does.  A performance monitor on the front panel, and an Arduino development board built in for other playing around.  When I get bored I'm going to remake the front panel and cut the holes for the extra Arduino pins.



Thursday, July 11, 2013

Software Installation Notes for New PC


This post is just a boring record of installing my software environment on my new desktop PC.  This is my new i7 4770K, dual video card, four monitor beast.  I use it for software development, electronics design and an occasional game.
http://blog.workingsi.com/2013/06/intel-haswell-4770k-pc-build.html

I'm back to running Windows 7.  I consider the Windows 8 experiment a failure on the previous machine.  8 is harder to use due to the extra menus and missing start menu, for no upside for the desktop user.

This is is my to do list and notes