Earthly

thanks

Well I'm not sure I understand what's going on underneath the hood yet, so I wouldn't be much help. But for starts, double negatives are always confusing, so changing "don't remove" to "keep" in the first sentence goes a long way for readability :slightly_smiling_face:

Please let me know how you would rewrite it :slightly_smiling_face:

yes

Hmm, the language of that flag is relatively confusing. Is it saying you would likely enable that option when not using --keep-ts?

You can also set this other parameter to remove certain meta-information from the cargo cache, and get partial cargo caching (but still performant) https://github.com/earthly/lib/blob/main/rust/README.md#keep_fingerprints-false

Interesting. So does this then require every Rust source file to be copied with this flag if we're the Earthly provided functions?

Earthly COPY by default overwrites file mtimes, unless passed that option I mentioned

Hi Josh, this is the explanation: our function lets you run cargo with some directories mounted so cargo its own cache (saved on such mounts) warm even when the earthly cache fails and the command is recomputed. The problem came because cargo caching algorithm checks file mtimes to detect changes, so even if you change the contents of a file, if its mtime is the same the cache entry would still be considered valid.

So using COPY --keep-ts src/lib.rs seems to solve the caching issue. However, using COPY --keep-ts --dir src does not work the same way. Is that a known issue?

Sure, I'll give it a test in a bit and get back to you. To answer your previous question: yes, I had a bug in my original attempt, and I later found out that it doesn't actually cachet the DO command. The actual caching is happening somewhere in the function. We do utilize a bit of the Earthly provided Rust library. Is that what you were testing against?

Ok, I think it might be related to the caching provided by cargo, that is in not detecting the changes due to the timestamps of the files being overwritten. Can you try copying the files with the option to keep their host timestamps? —keep-ts

Hi Joshua! the commands called from the function shouldn’t be cached after the copy one is not. Is this a figurative question? If not, can you provide a reproduction case?

<@U035VU5K46Q> should be able to answer this

COPY Cargo.toml .
COPY wasi+build-rust-bindings/hermes.rs src/hermes.rs

DO rust-ci+CARGO \
    --args "build --target wasm32-unknown-unknown --release" \
    --output="wasm32-unknown-unknown/release/ipfs_test_component.wasm"```
The details are not really important here, but let's say I change `src/lib.rs` and rerun this target. The `DO` command is fundamentally calling `cargo` which will use the code from `src` (as well as the other copied code) in order to produce a WASM artifact. The issue is that, while changing `src/lib.rs` does bust the cache and cause it to recopy it, the `DO` command remains cached. Meaning, even though the source code changes, the output artifact never does.

Is the only option getting rid of the `DO` in this case?