Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

dstack ls fails on macOS and Windows if an artifact contains "case-sensitive" symlinks #189

Closed
kengz opened this issue Feb 12, 2023 · 2 comments
Assignees
Labels
bug Something isn't working

Comments

@kengz
Copy link

kengz commented Feb 12, 2023

Describe the bug
Running dstack ls on large artifacts (Conda env in this case) throws an error: OSError: [Errno 62] Too many levels of symbolic links (see screenshot and logs below)

Version

  • The dstack CLI version 0.1
  • The operational system version Apple M2 MacOS 13.1
  • The Python version 3.10.4
  • (Optional) Other Python packages versions (pip freeze or conda list)

Minimal example
Source code kengz/lean-dl-example@f4c06f0

Steps to reproduce

  • dstack run setup-conda
  • wait for completion
  • dstack ls great-starfish-0 (or whatever the run name is)

Expected behavior
Shows the conda env folder

Logs

a82a3ec14d6f463cb76e83261598d865.zip

Screenshots
Screenshot 2023-02-12 at 4 11 08 PM

Additional context
Add any other context about the problem here.

@kengz kengz added the bug Something isn't working label Feb 12, 2023
@peterschmidt85
Copy link
Contributor

peterschmidt85 commented Feb 19, 2023

Additional context:

According to conda/conda#10333 (comment), the problem is caused by loading a package that is designed for a file system that is case-sensitive (i.e. Linux) onto a filesystem that is case-insensitive (i.e. macOS or Windows).

ls -l /Users/cheptsov/.dstack/artifacts/github.com/kengz/lean-dl-example/tasty-warthog-0,setup-conda,0/opt/conda/envs/dl/share/terminfo/N/ncr260vt300wpp
lrwxr-xr-x  1 cheptsov  staff  19 Feb 17 13:26 /Users/cheptsov/.dstack/artifacts/github.com/kengz/lean-dl-example/tasty-warthog-0,setup-conda,0/opt/conda/envs/dl/share/terminfo/N/ncr260vt300wpp -> ../N/NCR260VT300WPP

@peterschmidt85
Copy link
Contributor

Implementation notes:

Similar to ls in Linux, in dstack ls, for symlinks, the size should represent the size of the symlink and not the actual file. Thus, the command should use os.stat(..., follow_symlinks=False).st_size instead of os.path.getsize(...).

Additional information:

  1. Support symlinks in cached files #180 Currently, dstack may sometimes not handle symlinks properly. Though we haven't seen any issues related to symlink in Conda yet, this should be kept in mind.
  2. Make dstack ls non-recursive by default #131 There is a plan to change the default behavior of the dstack ls command to make it not recursive by default. In the context of this issue, it makes sense to change the dstack ls command to display if the file is a symlink.

@peterschmidt85 peterschmidt85 self-assigned this Feb 20, 2023
peterschmidt85 added a commit that referenced this issue Feb 20, 2023
peterschmidt85 added a commit that referenced this issue Feb 20, 2023
@peterschmidt85 peterschmidt85 changed the title dstack ls on large artifacts throws: OSError: [Errno 62] Too many levels of symbolic links dstack ls fails on macOS and Windows if an artifact contains "case-sensitive" symlinks Feb 21, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

3 participants