Sharing some tips on best practice of handling inbound files using file adapter in SOA Suite 11g PS3.
Oracle SOA 11g File Adapter, always pickup the file when it is created in the directory it is configured to look out. This is file & directory are configured using the JCA property “IncludeFiles” & “PhysicalDirectory” respectively.
That said, if the file being created in the directory is big enough or if the source system is slow or any network latency, the adapter will pickup intermediate file and try to translate to the target schema. This behavior will fault the BPEL process and result the inbound message will be “rejected message” .
So how we solve this behavior?.. There are lot of things we should consider before proposing the solution to this behavior including the attributes of the source system, location of the source sytem, SOA WL server/domain topology, File Adapter configuration, file format, size, file markers,target end system, target system location, integration pattern -sync/async/one-way/ etc. etc
To make things simple, i will consider the input file is simple xml in this part. Here are out-of-the-box option available with the SOA Suite 11g PS3.
Firstly, the approach is to set the “MinimumAge” property in the JCA Adapter configuration.
MinimumAge – This parameter specifies the minimum age of files to be retrieved. This enables a large file to be completely copied into the input directory before it is retrieved for processing. The age is determined by the last modified time stamp
Now our issue is partially solved by setting this parameter, but what is the “ideal value” of this parameter ? Let us consider we are setting this parameter to 300 (5 min).
At this time, 2 questions need to be answered
(1) Is it safe to assume the file will be created within 5 min in production ?
This might work well in unit/sys/uat environment, as it is less probably all the batch interfaces will be tested simultaneously and thus server load is minimum.
Is this “5 min” works at the production environment too where we expect multiple batch interfaces running parallely in a WL domain? If the content of the file is business critical like shipment details, quote info, pricing details, asset maintenance – no business user afford to leave these contents in the middle-ware layer as a rejected message.
(2) Is business afford to wait for 5 min, even if file got created in 1st minute itself ?
This is again customer’s decision on how the data is important for the enterprise.
Assuming the data is critical and the destination end system cannot afford to wait for 5 min, the other approach can be proposed is creating a addition TriggerFile with actual data-file. The interface should pickup data-file only when the source system creates the “TriggerFile”.
The below are some of the JCA property that you should set in file adaper JCA file – TriggerFilePhysicalDirectory, TriggerFile.
TriggerFilePhysicalDirectory- The directory path where the Oracle File or FTP Adapter looks for the trigger files.
TriggerFile-The name of the trigger file that activates the inbound Oracle File or FTP Adapter
By this approach, we can always ensure the data-file will be pickedup not only when it is complete but also as soon as it got created.
Hope this helps!