{"id":15116,"date":"2023-01-12T00:00:00","date_gmt":"2023-01-12T00:00:00","guid":{"rendered":"https:\/\/www.dmcinfo.com\/our-work\/test-data-centralization-standardization-and-storage-using-python-ni-systemlink-server\/"},"modified":"2025-05-29T22:47:55","modified_gmt":"2025-05-29T22:47:55","slug":"test-data-centralization-standardization-and-storage-using-python-ni-systemlink-server","status":"publish","type":"our_work","link":"https:\/\/www.dmcinfo.com\/our-work\/test-data-centralization-standardization-and-storage-using-python-ni-systemlink-server\/","title":{"rendered":"Test Data Centralization, Standardization, and Storage using Python & NI SystemLink Server"},"content":{"rendered":"

DMC helped the client deploy a solution centered around NI\u2019s SystemLink Server software.\u00a0 The SystemLink Server played several roles; it was the data aggregator from all test stands, the analysis engine of incoming data files, the long-term storage location of data files, and the asset monitor and software manager for the test stations.<\/p>\r\n\r\n

Our client had about a dozen identical data acquisition test stands deployed on their production floor that each executed a test sequence on a product and saved a data file locally to the PC. All of the data was saved long-term on each test stand, which made it difficult to retrieve data from a test stand or compare data between multiple. This also made them susceptible to data loss if a test station PC or a hard drive fails.<\/p>\r\n\r\n

DMC had to take a different approach, as we did not have the option to modify the test software directly to save the data to a central location rather than its C Drive. This was because the extensive validation for the customer\u2019s custom testing software had already passed, and they did not want to repeat the validation process.<\/p>\r\n\r\n

The customer was seeking a solution where the data from all test stations would be stored in one central location so that data from any test runs from any station could be quickly retrieved and analyzed.<\/p>\r\n\r\n

Python Windows Service<\/h3>\r\n\r\n

As we were not allowed to modify the test software directly, DMC wrote a Windows Service in Python that automatically moved new data files from each test stand to the SystemLink server.<\/p>\r\n\r\n

The service is installed on each test stand and monitors a particular directory on the test stand\u2019s C Drive for new data files. When new files are found, the service uploads them to the SystemLink Server using the server\u2019s REST API.<\/p>\r\n\r\n

SystemLink Systems Manager<\/h3>\r\n\r\n

The service was packaged into an NI Package using NI Package Builder<\/a>. Using SystemLink\u2019s Systems Management module, this NI Package was remotely installed on each test stand using SystemLink\u2019s web interface. This installation included registering the Python application with Windows as a Windows Service so that it is always monitoring for new data files.<\/p>\r\n\r\n

SystemLink Data Plugin<\/h3>\r\n\r\n

The raw data files from each station were custom-formatted CSV files. Each file could store\u00a0different information about the test and any number of different test steps \u2014 which would make directly comparing data from CSV files difficult.<\/p>\r\n\r\n

SystemLink Server facilitates the transformation of custom file types and formats to a common TDM<\/a> format using DataPlugins<\/a>. After conversion to TDM, data can be much more efficiently searched.<\/p>\r\n\r\n

DMC wrote a custom DataPlugin in Python that converted the client\u2019s custom-formatted CSV files into TDM files. Any CSV data file that is uploaded to the SystemLink Server would automatically be transformed into the friendlier TDM format.<\/p>\r\n\r\n

\"TDM<\/a>Analysis Automation Script<\/h3>\r\n\r\n

After converting that test data into a common TDM format, DMC created an Analysis Automation<\/a> script that automatically triggers for incoming data files. The automation script extracts test metadata and a list of \u201csteps\u201d that were executed for the test from the TDM file. Then, this information is sent to SystemLink\u2019s Test Monitor<\/a> module using the Test Monitor REST API.<\/p>\r\n\r\n

Test Monitor facilitates several useful functions for an engineer who may want to view past test results from a web browser:<\/p>\r\n\r\n

    \r\n
  1. Write queries to search for specific tests using test metadata (date and time, serial number, part number, operator, test station, etc.)<\/li>\r\n
  2. View a list of steps and step results for a specific test<\/li>\r\n
  3. Generate dashboards or reports for a quick view of the overall status of the testing systems.<\/li>\r\n<\/ol>\r\n\r\n

    \"Test<\/a><\/h3>\r\n\r\n

    Long-Term Storage<\/h3>\r\n\r\n

    Raw CSV and processed TDM data files are saved long-term on the SystemLink server. These files are directly linked to their test in Test Monitor so that original test data files are easily accessible, searchable, viewable, and downloadable.<\/p>\r\n\r\n

    Learn more about DMC’s\u00a0Test & Measurement solutions<\/a>, and contact us<\/a> today for your next project.<\/strong><\/p>\r\n","protected":false},"excerpt":{"rendered":"

    DMC helped the client deploy a solution centered around NI\u2019s SystemLink Server software.\u00a0 The SystemLink Server played several roles; it was the data aggregator from all test stands, the analysis engine of incoming data files, the long-term storage location of data files, and the asset monitor and software manager for the test stations. Our client […]<\/p>\n","protected":false},"author":8,"featured_media":15112,"template":"","meta":{"customer":"NI subcontracted us for this project; the end user was Cobham","summary":"

    DMC helped a client in the aerospace industry deploy a SystemLink<\/a>-based solution to meet their needs to aggregate, analyze, and store data from numerous test stations.<\/p>\r\n\r\n

    \"Flow<\/a><\/p>\r\n","description":"","customer_benefits":"