I have been working on designing a daily upload node.js application that calls a third-party in order to refresh the application MongoDB datastore. One of the solutions I was investigating was Heroku Worjer Dynos, Background Jobs and Queuing. I spent several hours of scratching my head trying to figure out how I could add a refresh button feature to my web application using the above pattern. I decided that it was not really a feature that should be implemented.
What I need to do:
- Create a refresh file by call third party service and saving the query results to a JSON flat file
- Drop the MongoDB collection. I don’t have Mongo available to me on the dyno so I will need to connect using headstone (Headless Keystone) script to purge all of the collections documents.
- Create a new MongoDB collection using the the JSON flat file as input. Again headstone (Hedless KeystoneJS) script to insert the documents.
- Remove the JSON flat file
With Heroku I will be able to create a One-Off Dyno and Schedule it on a nightly. Having Papertail enabled to monitor and notify me of any issues.