Summary
The text addresses challenges in executing commands that require SSH access to private module repositories, both locally and in GitHub Actions. It proposes solutions like passing passwords as arguments, using conditionals, and adjusting GitHub authentication settings. The author seeks alternative methods for improving CI authentication and shares a snippet for configuring GO environment parameters and caches. There is a discussion about embedding HTTPS git credentials in Docker layers, clarifying that credentials may appear in commands but are not saved. The author is interested in setting up a Go build within actions and asks about a repository for common Earthly functions, as well as the necessity of the --keep-ts
option for caching in Earthly, noting that Golang uses timestamps for caching, and without it, static timestamps can cause issues.
ingwar
and without it Earthly sets static timestamp on them and caches dont work (actually they break stuff, not working wouldynt be that bad)..
ingwar
Because golang uses timestamps on files for caching purposes..
brandon356
Also I am curious on the reason --keep-ts
is required when using a cache in Earthly
brandon356
Thank you! This makes sense now. I've been trying to figure out how to set up a Go build inside actions and was surprised Rust was the only language-specific docs in Earthly. Is there a repo/marketplace for public/common Earthly functions like this somewhere?
ingwar
RUN --secret GIT_USER=GITHUB_USER --secret GIT_PASS=GITHUB_TOKEN eval $cmd
and that line has credentials but they are never saved (at least as you dont save them as part of cmd)
ingwar
RUN git config --global credential.helper '!p() { printf "username=${GIT_USER}\\npassword=${GIT_PASS}"; }; p'
that line is saved as layer, but it dont have credentials..
ingwar
I wrote it so long ago that I had to loo whats there..
ingwar
nooo.. actually not..
ingwar
Actually yes.. but we always use it in our build images.. that are temporary..
brandon356
The issue with this approach is that you embed the https git credentials in a Docker layer right? <@U02P31BPR6X>
nicholas.thomson
Wow this is awesome. Thank you for snippets!
ingwar
Cause compile just works instantaneously..
ingwar
with that setup you dont need to do go mod download
dance..
ingwar
Only thing to remember (that we learned hard way) is that when you cpy something inside earthly and use cache you always need to add --keep-ts
ingwar
And if your using GO then life changing thing for us was
# Call example:
# DO commons+KTB_GO_PREPARE
KTB_GO_PREPARE:
FUNCTION
ENV GOCACHE=/go-cache
ENV GOMODCACHE=/go-mod-cache
ENV GOPRIVATE=<http://github.com/some|github.com/some>
CACHE --sharing=shared --id=kbt_go_cache $GOCACHE
CACHE --sharing=shared --id=kbt_go_mod_cache $GOMODCACHE```
ingwar
CI depending from setup use plain or ssh..
ingwar
Default config is ssh so and thats what most people use at local machines..
ingwar
We have something like that..
# Its mostly used for go programs that reach private repositories.
# Call example:
# DO commons+CONFIGURE_GITHUB --GITHUB_AUTH=ssh
# DO commons+CONFIGURE_GITHUB --GITHUB_AUTH=plain
CONFIGURE_GITHUB:
FUNCTION
ARG GITHUB_AUTH=ssh
ENV GITHUB_AUTH=$GITHUB_AUTH
IF [ "$GITHUB_AUTH" = "plain" ]
RUN git config --global credential.helper '!p() { printf "username=${GIT_USER}\\npassword=${GIT_PASS}"; }; p'
ELSE
RUN git config --global url."<ssh://git@github.com/>".insteadOf <https://github.com/>
RUN mkdir -p -m 0600 ~/.ssh && ssh-keyscan <http://github.com|github.com> >> ~/.ssh/known_hosts
END
# Run any FUNCTION with github configured
# Call example:
# DO commons+RUN_WITH_GITHUB --cmd="git clone <https://github.com/some/repo>"
RUN_WITH_GITHUB:
FUNCTION
ARG --required cmd
ARG DEBUG_PIPELINE=false
IF [ "$GITHUB_AUTH" = "plain" ]
RUN --secret GIT_USER=GITHUB_USER --secret GIT_PASS=GITHUB_TOKEN eval $cmd
ELSE
RUN --ssh eval $cmd
END
# Run FUNCTION with github configured
# Its mostly used for go programs that reach private repositories.
# Call example:
# DO commons+GO_WITH_GITHUB --cmd="go build"
GO_WITH_GITHUB:
FUNCTION
DO +KTB_GO_PREPARE
ARG --required cmd
ARG DESTDIR=/build/output
ARG DEBUG_PIPELINE=false
ARG GO_TEST_FLAGS
IF [ "$GITHUB_AUTH" = "plain" ]
RUN --secret GIT_USER=GITHUB_USER --secret GIT_PASS=GITHUB_TOKEN \
eval $cmd
ELSE
RUN --ssh eval $cmd
END```
nicholas.thomson
I just resorted to passing the password as an ARG, and then using a conditional to call -ssh
only if the ARG is not defined. It's a bit hacky, so still open to suggestions
nicholas.thomson
Hey earthly folks :wave:
I have a Earthfile with a target for running go mod download
that needs to pass through -ssh
so that go
can access my private module repositories. Everything works great locally! Now when I try to run that same step as part of the build process in a Github action, earthly fails with failed: no SSH key "" forwarded from the client
. I would expect this since GHA doesn't have an SSH socket running as far as I know. Is it possible to conditionally use SSH, or is there a better method for authenticating with private git repositories that can support being run in CI?