Developers

EDI Conversion File Watcher Service in C#

This is a sample C# command-line program which waits for file-change notifications in a directory, and upon seeing new EDI files, transforms them into XML files. There is a corresponding Java version of this program also.

EDI Service Details

EDI files are often dropped into a directory, with the intention that some program will iterate through them and convert them. It's possible that they were placed here by another program across a network share or via FTP, or another application on this same directory.

In order to ensure we don't start working on the file before the creator has finished writing it, we make the assumption that the file will be created under some name that does not end in ".edi". What we will trigger on is the Renamed event to an ".edi" file, since that is an atomic operation.

Because .Net includes file system monitoring services, we don't need to poll every n seconds. We can simply wait for events to notify our callback method OnRenamed.

If for some reason we see a Renamed event but there is already a corresponding ".xml" file, we do not convert, so that nothing is overridden.

If there are any errors, the partial ".xml" file is removed, and instead a file with the suffix ".bad" is created, containing the text of the error.

The program will terminate if a file called STOP is created or modified in the scan directory.

A log of the actions is written to the console. A sample of the log might look like:

2007-12-28T11:23:31 Converting c:\inbox\831.edi
2007-12-28T11:23:31 Converted c:\inbox\831.edi

EDI File Polling Service Implementation in C#

First, set up the using imports, namespace, and setup code for the Main method.

using System;
using System.IO;
using System.Text;
using DDTek.XmlConverter;

namespace DDTek.Tutorial {
class Scanner {
    static void Main(String[] args) {
        new Scanner(args[0]);
    }

Also, a small utility function to return the date/time for the logging output.

static private String now() {
return DateTime.Now.ToString("s");
}

We need to set up the ConverterFactory for later; the same one can be used to create converters throughout the life of the application instance.

Next, we set up two watchers. The first looks for any Renamed events, and when it sees them calls our OnRenamed method.

The second watcher looks to see if the STOP file is created or touched. We call the WaitForChanged method which blocks until the request events occur. When they do, our code simply exits the method &emdash; terminating our application.

private ConverterFactory cf;

Scanner(String dir) {
    cf = new ConverterFactory();

    FileSystemWatcher fsw = new FileSystemWatcher(dir, "*.edi");
    fsw.Renamed += new RenamedEventHandler(OnRenamed);
    fsw.EnableRaisingEvents = true;

    FileSystemWatcher fsq = new FileSystemWatcher(dir, "STOP");
    fsq.EnableRaisingEvents = true;

    fsq.WaitForChanged(WatcherChangeTypes.Created | WatcherChangeTypes.Changed);
}

The implementation of the C# OnRenamed callback that is triggered whenever a new EDI file is deposited is fairly straight-forward.

We need to weed out any notifications for files other than ".edi", and also those for which for some reason a corresponding ".xml" file already exists, so that we don't trash anything already there.

private void OnRenamed(object source, RenamedEventArgs rea) {
    String fedi = rea.FullPath;
    if (!fedi.ToLower().EndsWith(".edi"))
        return;

    String fxml = fedi.Substring(0, fedi.Length - 4) + ".xml";
    if (File.Exists(fxml)) {
        //Console.WriteLine("{0} Already handled {1}", now(), fedi);
        return;
    }

Now the task is to convert one. Notice how small the actual "convert" code is, relative to the size of the entire service!

Console.WriteLine("{0} Converting {1}", now(), fedi);

try {
    ConvertToXml c2x = cf.CreateConvertToXml("EDI");
    c2x.Convert(new UriSource(fedi), new UriResult(fxml));
    Console.WriteLine("{0} Converted {1}", now(), fedi);

The last portion of our program concerns cleaning up from error conditions. We log the error into a ".bad" file after removing the ".xml" file (which otherwise would be imcomplete). A quick message to the console lets the operator know something is awry. In a real program, there would probably be email notification or output to a logging service to catch that.

        } catch (Exception e) {
            File.Delete(fxml);
            Console.WriteLine("{0} Error on {1}", now(), fedi);
            String fbad = fedi.Substring(0, fedi.Length - 4) + ".bad";
            Stream fs = new FileStream(fbad, FileMode.Create, FileAccess.ReadWrite);
            TextWriter tw = new StreamWriter(fs, Encoding.UTF8);
            tw.WriteLine(e.ToString());
            tw.Close();
        }
    }
}
}

Future Expansion

Of course, this is only sample code for you to get ideas. Some areas for improvement might be:

Look for STOP in another directory that maybe isn't network-mounted, to keep a user from stopping the service by uploading a file called STOP
Moving the input files to a different place after processing
Putting the output files in different directory
Allowing different URI "EDI" options, perhaps based on filename
Automatically generate acknowledgements: ACK for HL7, 997 for X12 or CONTRL for EDIFACT/EANCOM/IATA

DotNet XML Converters as Components

This package can be downloaded as a Visual Studio 2005 solution from edi-watcher-dotnet.zip.

The key concept to remember is that DataDirect XML Converters are components that can be hooked into your applications wherever needed. Try them out and see how they can ease transformation of complex data by handling validation and conversion for you. Download XML Converters today!

Feeds

Multimedia

Newsletter