Use spot instances for extended tests

[blaginin]

I had spot=false because those runs were slow. Now they are quite fast, so we can save up while running them:

Before:

After:

Instance family differs between runs (runs-on picks one based on current cost/availability), but the numbers should be illustrative nonetheless.

I also put cpu to 32 since the tests are quite fast

[alamb]

Let's give it a try! Thank you @blaginin

Do you know what happens if we are running on a spot instance and it is killed/terminated? Does the job auto restart?

Looks to me like spot instances get a 2min grace period: https://aws.amazon.com/blogs/compute/best-practices-to-optimize-your-amazon-ec2-spot-instances-usage/?refid=ea118e8f-a658-4276-ba3a-0086cd46ef98

[blaginin]

yes, it will be auto restarted 🙂 in fact, that is already been happening to our Rust CI as those also use spot instances (when available)

[blaginin]

more on that:

https://runs-on.com/configuration/spot-instances/

In practice, spot instance termination is extremely rare for instances that have been running for less than 1h, making it ideal for all CI workflows that last less than 1h, and which do not trigger critical side-effects, such as deployments jobs.

[alamb]

more on that:

https://runs-on.com/configuration/spot-instances/

In practice, spot instance termination is extremely rare for instances that have been running for less than 1h, making it ideal for all CI workflows that last less than 1h, and which do not trigger critical side-effects, such as deployments jobs.

LOL -- my personal experience in the past is that spot instances are pretty stable most of the time, but then every now and again, for totally non transparent reasons, they are killed / moved / unavailable for a while before settling down.

[blaginin]

yeah, i saw (rarely) CI spot jobs are killed - then runs-on spawns a retry which is always on-demand not spot