I have a cron job that edits images, but because RAM memory is limited I would like to divide the task over numerous instances. To keep things safe I’m checking the request header for “X-Appengine-Cron”, which only Cloud Scheduler can set (it gets removed from outside requests).
The main cron job request divides the tasks and calls multiple new requests to divide the tasks over multiple instances. However, I obviously cannot pass on X-Appengine-Cron.
Ideally I would be able to make post requests to my app and manage it with a waitgroup (so I can manage how many instances are taken up)
How can I divide the cron job over multiple instances and keep it safe (i.e. with X-Appengine-Cron flag intact)?