I REPLACED MYSELF WITH A COMPUTER

I’ve always had a kinship with technology. Perhaps I’m bipolar… If you didn’t laugh at that gem I wouldn’t read much further – it gets worse. As a result of fate, be it good measure or good luck I ended up in very technological career. Spending hours reading through Radio Shack catalogs at a young age must have built thousands of synapse in my brain all geared for remembering the current rating of a light bulb and other completely useless information. I’m positive my wife would have preferred I forged some synapses for remembering the garbage goes out on Monday morning, but that isn’t the way it happened. Of course, If I had been taking out the garbage as I was supposed to, perhaps I wouldn’t have been reading an artificial about the advancements in machine learning and artificial intelligence which led me to my ultimate problem. I replaced myself with a computer.

Computers have infinite memory, or at least more than my squishy grey matter. The synapses I developed for remembering this information pale in comparison to what my digital counterpart is capable of. Since the technical components of my job are basically memory… Why shouldn’t the cold steel box on my desk preform exceptionally? This sounds like a great idea, right? I’ll take the repetitious technical parts of my job, and have little blue do it! So that’s what I did.

I can’t really reveal the secret sauce here but I was able to train my pet mike (I’m going to call my computer ‘mike’ with a little m from now on) in about 15 hours. This felt like the ultimate insult. It took me 40ish years to build an archive of awesome including many bumps and bruises. My digital doppelganger did it in just 15 hours. I’m not saying mike is perfect, but he scored at 70% accuracy on the training set, and he is the very first mike I’ve ever built.

Insult aside, I was elated. I had replaced myself with a computer! Wait…. I replaced myself with a computer. If your self-worth hangs on the pride you put into your job, you may want to sit down for the next part. Like so many people who thought their job was untouchable, bankers, journalists, printing press repair technicians, I am replaceable. Worse, I replaced me in just 15 hours.

You think many things after you replace yourself with a computer, can I bury mike late at night so no one will know? How long until someone else builds a mike or a stacey or a bruce and I’m out of a job? What if mike puts hundreds of people out of work? Did I feed the dog?

I’ve written thousands of lines of code in my life. I’ve automated and simplified tasks for people, and enriched their lives as a result. I’ve never been faced with such a moral dilemma and I hope you never will. Unfortunately, Pandora’s box is open, and mike isn’t going away. Prepare yourself.

If you took the time to read this, I hope you enjoyed it, mike did.

Original Posting – LinkedIn January 25 2017

The Path to Connected Machines

Could you run your business without email? It’s an easy question to answer today, but could you have answered the same question as easily in 1994 before email exploded and became commonplace? The same is true with much of the technological advancement of this century; the PC, the cell phone, the smartphone. Many managers will remember desperately trying to justify Blackberry smart phones for mobile workforces. Unfortunately, the advantages sermonized by these managers largely fell on deaf ears as the return on investment wasn’t clear in the early days. Years later the smartphone is equally as important as eye glasses to some people, not being able see without either of them. However, many would still be challenged to put a dollar value to the benefit that the smart phone brings. The emerging technologies in manufacturing are of the same creed.

    Certainly connected machines will change the way products are manufactured just as the PC and smartphone have changed the way businesses run.

The quickly attainable result and arguably the reason every machine should be connected today is to replace inefficient manual processes. In many machining factories, line supervisors are sent out three times a day to collect parts counts with a clipboard. The collected count is transferred to an excel spreadsheet which in turn is used to evaluate the output of the machine and production line. Additionally, when a machine breaks the operator must search out the line supervisor who invariably files a maintenance requisition to notify the maintenance department. These manual processes that happen regularly within machining factories can be automated on networked machines which quickly improves both the accuracy and the speed of the process. Finally, in collecting the parts count and machine status information and combining it with quality reports, we can understand the operational efficiency per machine and identify production issues.

Although the low hanging fruit is simply collecting and reporting data that is manually being complied today, it is not the greatest benefit to most organizations and certainly not the endpoint. Instead it is the starting point which validates running copper to each machine and investing in infrastructure to support a connected factory. The greater benefits are realized after the data is being collected. For example, ERP systems are common place and allow employees to process business transactions in an auditable way. By connecting machines to ERP systems it provides visibility to inbound, outbound and in process inventory levels, demand vs. production rates and actual vs. quoted cost. With this high level of visibility in both process and accounting more accurate information can be shared with customers, and better decisions can be made by all stakeholders.

As demonstrated above, it doesn’t require a leap of faith to realize the immediate benefits of connected machines. Common sense and planning will shift the production floor to a connected version of its former self that provides ample visibility to make more informed decisions. The next evolution beyond data gathering further leverages the data produced. The databases that store all of the historical data from the machines will be analyzed using any number of business intelligence (BI) software solutions. The very active market for BI software has incredibly versatile features such as natural language query, automated insights, trend analysis, outlier identification, dashboard, reporting, and so on. The data analysis will become a regular if not daily process on the shop floor as it is the most effective way to both generate and answer questions about the parts, process and business. The connected machines have closed the continuous improvement loop with incredible accuracy and speed.

One of the most ambitious views of a connected factory is the absence of people as machines speak directly to other machines to control the manufacturing process. Although this dystopian version of a connected factory has roots in reality the ultimate efficacy has yet to be proven. Instead, CMM machines or gauges that update CNC machines free operators to evaluate issues with the tooling which results in fewer scrap pieces and a more satisfying purpose for the operator. Cameras that track parts throughout the process and prevent running a tap without confirming the drill operation is complete reduce tedious rework and subsequent tool repair. A fork lift operator can plan the best route based on actual parts accumulation fed directly to the vehicle thereby reducing the miles she must drive. When machines are connected all stakeholders benefit, including shop floor team members.

As machines are connected to sate the flow of data for factory processes we can then start integrating statistical analytics and really elevate the value in connected machines. Techniques long used in analyzing financial markets and weather systems are adapted and used to quantify the mechanical conditions in a machine and predict mechanical failures. Parts quality indicators can be measured during machining and the statistical probability of good vs a bad part instantly actioned upon. Separated by only one degrees of complexity from statistical analysis is machine learning which when employed can predict a failed electrical or mechanical component, accurately predict why a machine is down, and ultimately learn the best control algorithm for producing the most accurate part in the quickest manner. Importantly, artificial intelligence isn’t just science fiction. All of these pieces of a connected factory exist and are deployed today. The technology and skill set has just reached a point where it is accessible en masse.

Connected machines will invariably change the way products are manufactured. Understandably the investment in a connected factory needs to be justified and carefully navigated; however, you don’t need to look past the smart phone you carry to see the importance of a connected factory.

MTConnect in 2016

Two years ago, when my research into MTConnect began, the landscape of industrial data collection was far different than it is today. There were a few Amazon, Google, Salesforce, IBM type companies driving deliverables online with a very small percentage profiting from the often large investments in data centers, and programming. Now, all industries from your local grocery store, banks and high tech startups are all driving to a singular goal, drive customer interaction online as either self-directed or end point delivered. Not only has the value in owning data been realized, but the costs in not owning data are being recognized. The need for large data viaducts, flexible processing power, and reliable, responsive data storage all while being instantly scalable created a business in cloud computing, and evolved many services to capitalize on business opportunity and necessity. Revisiting MTConnect today reveals a different use case than even two years ago.

The influx of switchboard applications such as nifi, fluentd, and flume has diminished the requirement for standards in machine interconnection. Almost any protocol can be made interoperable with any other protocol. Moreover, industry specific standards are now detraction from the myriad of possibilities for data analytics and big data logistics freely available to everyone as open source projects. MTConnect should now be recognized as an on premise connector rather than an end to end solution. This model is far more suited to modern business intelligence than a proprietary MTConnect endpoint. Importantly, MTConnect could then be looked at as the ‘universal’ adapter for IIoT to BI property because of its interoperability with nifi, fluentd and flume, etc.

Even though the standardization through the use of a schema is depreciated, many parts of MTConnect serve a solid anchor to base extract, transform and load (ETL) operations from. Specifically, the state buffer created by the MTConnect agent greatly reduces the data throughput required. Additionally, the trapped ‘UNAVAILABLE’ state assists with tristate data that can be on/off/unknown. Furthermore, through the implementation of XSLT stylesheets available to implement in MTConnect; JSON as well as the native XML are available for consumption to upstream processes greatly simplifying the ingress of data from shop floor to cloud infrastructure.

Most certainly, the driving factor behind using MTConnect in shop floor monitoring is no longer the standardization to a communication protocol. Instead, the decision made long ago to open source the reference material will carry far more weight, as the race to data dominance ends in one place… at the bottom.

MTConnect Agent – FANUC Macro Variables

Macro variables are used in just about every shop process because of their flexibility and utility.  Being able to read them is a critical part of any data collection system.  As MTConnect focuses on standardization, the variables are excluded.  It is however simple to add them to any MTConnect connection to a FANUC CNC control.  Once the Adapter is setup to read macro variables ( see tutorial here) It’s time to setup the Agent.

The MTConnect agent setup is surprisingly simple.  Almost too simple.  Initially I struggled for hours trying to learn and understand how to extend the agent schema files to accept the macro variables.  I used this tutorial here as a base.  It turns out the only necessary step is to add the following line to the device xml file…

VMC-3Axis.xml

<DataItem type="VARIABLE" category="EVENT" id="mc1" name="whale">

That’s it…  That’s That and That’s all.  The type and id can be anything you like.  In this example I’ve used VARIABLE” and “mc1“.  The agent simply matches the value of the name property, with the matching name that is output from the adapter. 

adapter.ini

[adapter]
port = 7878
service = MTC Focus 1

[focus]
host = 10.211.55.5

[macros]
whale = 500

[pmc]
SspeedOvr = 30
Fovr = 12

Macro variable 500 is tied to  the name “whale” at the adapter.  The agent matches the name “whale” and continues to pass it on to the client.

macro screen

Too easy.

MTConnect Blog Posts

MTConnect Links

As the number of posts increases, some are getting lost.  Just to keep things organized here is a list of posts grouped by topic.

Machine

FANUC FS0iD FOCAS Setup

Adapter

MTConnect FANUC Adapter on Ubuntu Linux

MTConnect Adapter for Windows

MTConnect FANUC Adapter PMC Addresses

MTConnect-FANUC Macro Variables

Agent

MTConnect Agent on Ubuntu Linux

MTConnect Agent – FANUC Macro Variables

Client

This would be a great place to make any requests for tutorials or posts.  I’m happy to entertain.  Of course the target should be MTConnect :)

MTConnect – FANUC Macro Variables

Updating the FANUC Adpater for Stable Macro Variables

The current edition of the FANUC MTConnect adapter as of this writing was unstable on my platform.  Any time a macro variable was entered in the adapter.ini file, the adapter crashed shortly after connection.  This seems to be related to threading and access to the datum.  To rectify this I rewrote the getMacros() function of fanuc_adapater.  It no longer supports the multipath macro’s and is less efficient, however it works consistently without any errors.  Additionally, I was feeling lazy and included the <math.h> header to properly calculate the decimal point in the variable.

fanuc_adapter.cpp,  rewritten getMacros():

#include <math.h>
 void FanucAdapter::getMacros()
 {
    if (!mConnected)
    return;
    for (int i = 0; i < mMacroSampleCount; i++)
    {
       ODBM macro;
       short ret = cnc_rdmacro(mFlibhndl, mMacroSample[i]->getNumber(), sizeof(ODBM), &macro);
       if (ret == EW_OK)
       {
          double rational = macro.mcr_val;
          double decimal = macro.dec_val;
          double exp = pow( 10, decimal );
          double resultant = rational / exp;
          mMacroSample[i]->setValue(resultant);
       }
       else
       {
          printf("Could not retrieve PMC data at %d for %s: %d\n", mMacroSample[i], mMacroSample[i]->getNumber(), ret);
       }
    }
}

Now if we put a macro variable in the adapter.ini file as such:

adapter.ini

[adapter]
port = 7878
service = MTC Focus 1
 
[focus]
host = 192.168.82.15
 
[macros]
whale = 500
cabbage = 501
 
[pmc]
SspeedOvr = 30
Fovr = 12

the adapter will correctly pass on the named macro variables:

http://adapter.ip.address:7878/

2014-07-14T17:04:00.057Z|avail|AVAILABLE|part_count|7|whale|1.23456789|cabbage|2|SspeedOvr|0|Fovr|0|turkey|0|tool_id|0|program|3.0|line|0|block|O0000%|path_feedrate|0|path_position|0.0000000000 0.0000000000 0.0000000000|active_axes|X Y Z|mode|MANUAL_DATA_INPUT

2014-07-14T17:04:00.057Z|servo|NORMAL||||

2014-07-14T17:04:00.057Z|comms|NORMAL||||

2014-07-14T17:04:00.057Z|logic|FAULT|100|||PARAMETER ENABLE SWITCH ON
2014-07-14T17:04:00.057Z|motion|NORMAL||||

Which matches our FANUC FS0iD control:

macro screen

Cool Eh?  Macro variables are a critical part of any modern CNC shop, and being able to read them via MTConnect opens tremendous possibilities.  Custom parts counts can be made, operation conditions can be read,  etc.  The possibilities are endless.

MTConnect FANUC Adapter PMC Addresses

The current version of FANUC MTConnect adapter only supports PMC G address.  From a standardization point of view this is a great decision. The other PMC addresses including R,K,D,C and T are used by the machine tool builders and are assigned differently between machines.  Conversely the G Addresses are defined by FANUC and consistent between machines.  However, in practical terms, much can be gleaned about the machine condition through these addresses when customized to the machine.  With just a small modification to the existing FANUC MTConnect adapter, it can support all addresses.

By default, the addresses are defined in the adapter.ini file under the [pmc] section.  The sample adapter.ini file has two addresses mapped, G12-Feedrate override and

Firstly, the source file fanuc_adapter.cpp has one line of code that reads pmc addresses over FOCAS.  The FOCAS call expects the address type as an integer, G=0, F=1, Y=2, X=3, and so on.  In order to read addresses other than G we will need to specify the address.  The easiest was to achieve this is to pass an append the integer of the address to the front of the address.  For example, pmc address Y0078 would become 20078.  Then it’s just a simple matter of separating the prefix from the address.  Additionally, the orginal adapter incorrectly handles a negative value.  The code below is correct so the PMC address is reported properly if it is negative return. Here is the code:

fanuc_adapter.cpp
void FanucAdapter::getPMC()
{
 if (!mConnected)
 return;
 
 for (int i = 0; i &lt; mPMCCount; i++)
 {
 IODBPMC buf;
 
 // Seperate the data type
 int pmcType = mPMCAddress[i]/10000;
 int pmcAddress = mPMCAddress[i]%10000;
 
 short ret = pmc_rdpmcrng(mFlibhndl, pmcType, 0 /* byte */,
 pmcAddress, pmcAddress, 8 + 1,
 &buf);
 if (ret == EW_OK)
 {
 if (buf.u.cdata[0] < 0)
 mPMCVariable[i]->setValue(-buf.u.cdata[0] + 128);
 else
 mPMCVariable[i]->setValue(buf.u.cdata[0]);
 }
 else
 {
 printf("Could not retrieve PMC data at %d for %s: %d\n",
 mPMCAddress[i], mPMCVariable[i]->getName(), ret);
 }
 }
}

Now that the source code will properly read any PMC address, we just need to modify the addresses in the adapter.ini file as below:

adapter.ini:
[adapter]
port = 7878
service = MTC Focus 1
 
[focus]
host = 192.168.82.15
 
[macros]
 
 
[pmc]
# PMC Types G=0, F=1, Y=2, X=3, A=4, R=5, T=6, K=7, C=8, D=9
# PMC Address G22 would by 00022, R99 would be 50099 etc.
 
SspeedOvr = 00030
Fovr = 00012
turkey = 50033

This creates the following stream output from the MTConnect FANUC adapter:  Address R33 was 10000000 in binary, which correctly output as turkey|127 in decimal.

output of adapter at  http://some.ip.address:7878/
2014-07-10T15:12:52.532Z|Zoverheat|NORMAL||||
2014-07-10T15:12:52.532Z|Zservo|NORMAL||||
2014-07-10T15:12:52.633Z|part_count|7|SspeedOvr|0|Fovr|0|turkey|127|tool_id|0|program|3.0|line|0|block|O0000%|path_feedrate|0|path_position|0.0000000000 0.0000000000 -0.0010000000|mode|MANUAL_DATA_INPUT
2014-07-10T15:12:52.633Z|logic|FAULT|100|||PARAMETER ENABLE SWITCH ON

That’s it.  This same procedure will work for all controls including the FANUC FS30/31/32 and FS16/18/21 controls.  However, the address to number conversions may have to be re-arranged.

If you use this post please write a comment below.  Thanks.

MTConnect for FANUC Overview

 

 

When first starting developing with MTConnect all of the pieces can be very overwhelming.  I thought it might be helpful to break down the pieces and the connection between them.  Even though the MTConnect Adapter for FANUC and the MTConnect agent are complete from the GitHub repository, when developing client applications or setting up the pieces an overview is helpful.

overview

Click the picture for a larger view

Devices

Setting up MTConnect starts at the FANUC control.  The control must have an Ethernet connection and the optional FOCAS function.  Most modern controls have FOCAS available from the factory via an embedded Ethernet port on the main board of the CNC control.  Check this blog post here to setup the FOCAS connection.

The adapter and agent are best run on a server based on either Linux or Microsoft Windows.  Additionally, running the adapter on a low cost Linux platform located directly in the CNC can help reduce server and network load.

Communication

The FANUC control only speaks FOCAS, a very robust and powerful API that personal computers use to read and write information on the CNC.  The MTConnect adapter does all the heavy lifting and converts FOCAS to an MTConnect data stream.   The adapter streams data to the agent via http protocol which is human readable from Internet Explorer.  Here is the sample output from the FANUC adapter:  http://adapter.ip.address:7878/

2014-07-08T20:45:54.373983Z|system|NORMAL||||
2014-07-08T20:45:54.373983Z|Xtravel|NORMAL||||
2014-07-08T20:45:54.373983Z|Xoverheat|NORMAL||||
2014-07-08T20:45:54.373983Z|Xservo|NORMAL||||
2014-07-08T20:45:54.373983Z|Ytravel|NORMAL||||

Collation

As all of the pieces of MTConnect are based on some type of TCP communication, the devices find each other by knowing the IP Address and port number of the previous device.  The client knows the IP address of the agent, the agent knows the adapter, and the adapter knows the FANUC CNC.

Finally

Ultimately the client will consume XML requested from the agent.  The structure of the XML is determined by the schema specified for each machine.  The agent takes adapter data and matches the schema name‘s with adapter stream labels.  It records the stream into a buffer.  It is that buffer of data that is served when the client requests the XML.

 

MTConnect FANUC Adapter on Ubuntu Linux

 

The FANUC FOCAS shared library from the FANUC FOCAS CD A02B-0207-K737 version 4.1 or higher  must first be installed and registered in Ubuntu 14.04 LTS.

$ sudo cp libfwlib32.so.1.0.0 /usr/local/lib/libfwlib32.so.1.0.0
$ sudo ldconfig
$ sudo ln -s /usr/local/lib/libfwlib32.so.1.0.0 /usr/local/lib/libfwlib32.so

Next up we need to get the MTConnect adapter from GitHub.

$ cd ~ 
$ git clone https://github.com/mtconnect/adapter.git

We only actually need a limited subset of files from the adapter downloaded from GitHub.  For convenience in building the binary we will copy all the needed files to the same directory. (Note, so wildcards can be used more than the needed files are copied.)

$ mkdir fanuc
$ cp ~/adapter/fanuc/adapter.ini ~/fanuc/adapter.ini
$ cp ~/adapter/fanuc/fanuc.xml ~/fanuc/fanuc.xml
$ cp ~/adapter/fanuc/*.cpp ~/fanuc/
$ cp ~/adapter/fanuc/*.hpp ~/fanuc/
$ cp ~/adapter/src/*.cpp ~/fanuc/
$ cp ~/adapter/src/*.hpp ~/fanuc/
$ cp ~/adapter/minIni_07/*.c ~/fanuc/
$ cp ~/adapter/minIni_07/*.h ~/fanuc/

Once the files are in the ~/fanuc/ directory, we need to modify the source code.  The GitHub adapter was meant for Windows, and their are several functions that need to be modified.

$ sudo nano ~/fanuc/fanuc_adapter.cpp
  
    Remove: #include <excpt.h>
    Change: __try and __exception to try/catch(...)

    Change: Sleep(5000) to sleep(5)

    Change: _strnicmp() to strncasecmp()

    Add Before : short ret = :: cnc_allclibhndl3...

    long level = 3;
    std::string filename = "focas.log";
    const char * c =  filename.c_str();
    short log = ::cnc_startupprocess(level, c);

    Add After: cnc_freelibhndl....

    cnc_exitprocess();

Finally the header file for Linux from the FOCAS cd is copied to the directory.  Note the name change required as the source files refer to to Fwlib32.h.

$ sudo cp fwlib32.h ~/fanuc/Fwlib32.h

With the source code modified, its time to compile the binary.  Sorry for the sloppy g++ command, this could be cleaned up with a nice CMakeLists.txt.

$ cd ~/fanuc/
$ g++ minIni.c device_datum.cpp fanuc_axis.cpp fanuc_path.cpp service.cpp condition.cpp cutting_tool.cpp string_buffer.cpp logger.cpp client.cpp server.cpp adapter.cpp fanuc_adapter.cpp FanucAdapter.cpp -lfwlib32 -lpthread -o adapter

Finally setup the adapter.ini file with the appropriate settings for your machine and run the binary.

$ ./adapter debug adapter.ini

Conclusion I’m certain one day the source code for the Linux FANUC adapter will be available from GitHub as the code is just a slight adaptation from the Windows version.  Until then, I hope you enjoyed this tutorial!