We have a hosted instance of intersystems-cache. There is an aggregate log that is near 600 million rows and is rather abstract and complicated. I am planning to parse this log and store more meaningful events in an on-site database for easier reporting. I'm trying to figure out what my options are.
The only way I can access the data is over a VPN using ODBC or JDBC and Caché's SQL implementation.
What I'm looking for:
1. Middleware solution.
a. Scripting solution? VBScript, Python, Node?
b. 3rd Party Software? Some sort of Service Bus or similar?
c. Other options?
2. Target Database. Postgre? MySQL? Other? Needs to be free.
Regarding the "Scripting Solution", obviously I'm not looking for someone to write the script. I just want to know if this is a feasible approach. I have actually started down this path with VBScript with limited success. I was just hoping to find better options.
I looked for a "transformation" tag or similar but couldn't find it. Sorry if the tags or wrong or if this simply doesn't belong on dba.se.