E

Satellite Performance Issue with GHA Runner

Summary

The user observes that their satellite slows down when running more than two activities at the same time and is seeking a solution to implement round-robin scheduling between their available satellites while using the GHA action runner.

Status
resolved
Tags
    Source
    #earthly
      k

      kieran.mann

      8/29/2024

      I suppose a better "LB" could be implemented by calling earthly sat inspect <sat-name> for each of your "replica" satellites and then making decisions about which to use based on sleep/wake state and op load.

      Would be subject to timing issues if you have concurrent pipelines running that both make a decision about what sat to use before launching work but these tend to be the limitations of load balancing on the client side in my experience :slightly_smiling_face:

      n

      nacho

      8/29/2024

      Also consider vertically scaling them in the short term

      n

      nacho

      8/29/2024

      I would try to subdivide the job into smaller ones and assign each of them to a satellite as appropriate

      n

      nacho

      8/29/2024

      (Having a satellite awake but picking another that is asleep)

      n

      nacho

      8/29/2024

      That looks like a nice approach Kieran, but it might have some undesired effects in terms of satellite hibernation.

      k

      kieran.mann

      8/29/2024

      Interesting idea. I'm just starting to use GHA so I am by no means an expert but I suppose a simple client-side round robin could be implemented by: • Naming n satellites with a convention my-sat-0 , my-sat-1 etc • Adding an action at the beginning of your workflow to determine the satellite ◦ Getting a counter for the build, perhaps GITHUB_RUN_NUMBER ◦ Determining which sat to run on with GITHUB_RUN_NUMBER % n ◦ "Outputting" a value of the satellite to run on to use in the subsequent earthly stage(s)

      e

      e

      8/29/2024

      we notice that when our satellite is running more than two activities simultaneously it slows down - how could we round robin between our available satellites? we are running via the GHA action runner