Actually test AWS and GCP server in CI

[djmitche]

This will require having some limited credentials for these services available as secrets in GitHub. And to do so I think it would be appropriate to use GBF accounts, rather than my personal accounts. @tbabej what do you think -- would it be hard to set up AWS and GCP accounts? The testing would cost pennies per month.

[tbabej]

I can definitely look into that! One potential worry that I have is that if every test run in the CI costs us, it does open us up to potential attack vectors (i.e. someone spamming PRs/commits), but hopefully it would take a lot of pushes and MRs to make that a significant cost 🙂

[djmitche]

The storage cost is the big part, and the tests clean up the storage after each run, so that doesn't accumulate. The transfer costs for a few tiny bits of data are negligible. And by "big" I mean $0.05 or something. I think you can set spending limits in both clouds, too, so we could stop if it exceeds $5/mo for example.