Summary
The text addresses a problem with Earthly not utilizing the /etc/hosts file for hostname lookups, resulting in errors for certain hosts. It also notes the ability to reuse cache between CI runs using Earthly Satellites for improved performance. The discussion emphasizes the difficulty of maintaining reproducibility in builds due to DNS resolution issues and the need to treat builds as separate from the host environment. The user expresses a similar concern and questions where in Earthly the logic exists that disregards /etc/hosts during image operations, clarifying that it is not a docker-in-docker scenario.
sweaver
We are encountering a similar issue - where is the logic in earthly that ignores /etc/hosts when pulling/pushing images? It’s not a docker-in-docker situation, right?
nacho
True, builds are not completely reproducible since they might have external dependencies (for example a DNS server), and also true that IP resolution is not that crazy to be inherited from the host, but I find conceptually easier to understand thinking of any command as running in a build environment isolated from the host (not only RUN ones)
sakurainds
<@U035VU5K46Q> but you can't make any assumption that disregarding /etc/hosts results in a reproducible build.
I may be over-riding a dns result, precisely to achieve reproducability. For example, using an internal host name that is run over a VPN, a dev may need to set a particular IP to hit the correct machine, because their caching local resolver doesn't know about that host. (or in this case making a hostname resolve that can never resolve from anything past a local caching resolver because *.test is a reserved domain address)
nacho
I see, yes I was thinking of a RUN command. My argument about reproducibility remains though: Earthly is designed to run builds exactly the same no matter what machine you're running them form (yours, a team mate's, a satellite...)
sakurainds
I think we may be talking at cross purposes.
The exact line in my earthfile thats failing is:
SAVE IMAGE --push --insecure registry.cluster.test/cat-voices-docs:latest
not RUN. Yes, I would expect RUN to behave however the container is configured and ignore the hosts /etc/hosts file unless its explicitly copied into the container or run LOCALLY as you suggested.
But SAVE IMAGE , I think, should behave as if its run LOCALLY. It shouldn't run "inside" the container its attempting to save to the registry.
nacho
Now, for https://docs.earthly.dev/docs/earthfile#locally|LOCALLY targets then yes, your host file should be honored
nacho
Ok, maybe "portable" is not the word, but "reproducible".
You can always pass the hosts mappings through args or COPY commands to the build environment, and use those values to update the host file within the RUN command
sakurainds
There are lots of reasons why one might want to alter a DNS entry through /etc/hosts when doing local development.
sakurainds
<@U035VU5K46Q> That's very surprising, and I don't agree that builds wouldn't be portable. I can easily go into my upstream resolver and mess with the DNS lookup as well, but thats not detected by Earthly. Seems odd behaviour and I never saw it documented anywhere.
nacho
If you refer to the file in your host machine, then yes, it is. Otherwise the builds wouldn't be portable.
sakurainds
Using default tag: latest
The push refers to repository [registry.cluster.test/nginy.later]
14773070094d: Pushed
7d2fd59c368c: Pushed
56f8fe6aedcd: Pushed
9f4d73e635f1: Pushed
747b290aeba8: Pushed
fc1cf9ca5139: Pushed
5d4427064ecc: Pushed
latest: digest: sha256:0e1ac7f12d904a5ce077d1b5c763b5750c7985e524f6083e5eaa7e7313833440 size: 1778```
Also works OK with hosts defined in `/etc/hosts` .
sakurainds
For the avoidance of doubt:
* Host registry.cluster.test:80 was resolved.
* IPv6: (none)
* IPv4: 192.168.58.10
* Trying 192.168.58.10:80...
* Connected to registry.cluster.test <tel:(192.168.58.10|(192.168.58.10>) port 80
> GET /v2/ HTTP/1.1
> Host: registry.cluster.test
> User-Agent: curl/8.7.1
> Accept: */*
>
* Request completely sent off
< HTTP/1.1 200 OK
< Content-Length: 2
< Content-Type: application/json; charset=utf-8
< Date: Wed, 29 May 2024 05:39:40 GMT
< Docker-Distribution-Api-Version: registry/2.0
< Vary: Origin
< X-Content-Type-Options: nosniff
<
* Connection #0 to host registry.cluster.test left intact
{}⏎ ```
sakurainds
Is it normal that Earthly doesn't respect /etc/hosts for hostname lookups?
output | [----------] 100% exporting outputs
Error: failed to do request: Head "<https://registry.cluster.test/v2/cat-voices-docs/blobs/sha256:57e0e85367e114e9012cfa0cc7f3a815f90441bd011418289516dae791fb287a>": dial tcp: lookup registry.cluster.test on 192.168.69.1:53: no such host
🛰️ Reuse cache between CI runs with Earthly Satellites! 2-20X faster than without cache. Generous free tier <https://cloud.earthly.dev>
...
❯ getent hosts registry.cluster.test
192.168.58.10 registry.cluster.test```