Data Feed Management
DEGREE 53 FEED MANAGEMENT SYSTEM
When completing projects for our clients, we found that data feeds were often required and processing them separately for each individual solution was proving to be extremely time consuming.
In order to decrease the time taken to implement each data feed, we designed our own in-house Feed Management System. The feed management system contains a library of standard tasks used to perform common functions (such as obtaining data from a URL or reading content from a file).
The feed management system also allows us to deploy to the cloud, which can be used to reduce the load on a client's infrastructure and therefore increase stability. It also allows new tasks to be added and be performed together as a flow, for example:
- Obtain data from one source and convert it to another format for use by a mobile device.
- Obtain data from one source, convert it, restructure it and reorder the data for display.
- Obtain data from a file or queue, convert it, restructure it and reorder it for display.
- Aggregate data from two separate sources, merge it and then format it for use by a mobile device.
- Periodically delete unwanted files.