Skip to content

Conversation

@withered-magic
Copy link

@withered-magic withered-magic commented Jan 27, 2026

Fixes: #62

Adds support for installing gems from Git by including the Git sources in vendor/cache. Currently this uses git directly, but I'm planning to follow this PR up with another PR to use @sushain97's approach of downloading source archives from GitHub with the Bazel downloader.

Right now doesn't support Windows as I wasn't able to get this working with my limited Windows knowledge :(

AI Disclaimer

Core logic was written by me, only used AI for comments, setting up the example scaffolding as well as some of the more tedious parsing tests.

@@ -0,0 +1,29 @@
# frozen_string_literal: true
require 'fileutils'
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would it be reasonable to add a high-level comment on what this file does?

I think it's basically rsync'ing gemspec/extconf.rb files?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

just added a comment but yep exactly! we need this rsync-style functionality when there are Git gems because I noticed that there's some logic in Bunder / some gems that tries to normalize these files, which runs into "file is read-only" errors since it eventually tries to rewrite the (read-only bc of sandboxing) files from vendor/cache, so I need to make a copy of vendor/cache so that Bundler can rewrite stuff as it wants.

So to get around this I needed to make another copy which preserves symlinks except for those files that can potentially get rewritten by bundler (so far I've only noticed this happen for gemspecs and extconf.rb)

2.3.0
"""

# Sample Gemfile.lock content with multiple Git gems from same repo
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure these comments add much, I might just drop them?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(similar observation for a lot of the comments below, my guess is they're AI remnants)

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yep, I'll remove these!


raise 'must provide both srcdir and destdir' unless ARGV.length >= 2

$srcdir, $destdir = *ARGV
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a particular reason to use globals here vs something a bit more idiomatic, i.e. logic nested under if $PROGRAM_NAME == __FILE__

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

just refactored it to make it a bit more idiomatic and remove the globals!

_git_reset(rctx, git_repo)
_git_clean(rctx, git_repo)

git_metadata_folder = git_repo.directory.get_child(".git")
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do we delete this? Worth a comment

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll add a comment! Was mostly since this shouldn't be used by Bundler down the line and therefore didn't want it end up as an unnecessary input to the bundle install action later

if remote_name.endswith(".git"):
remote_name = remote_name[:-4]

extracted_path = "%s/%s-%s" % (cache_path, remote_name, git_package.revision[:12])
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I assume the 12 here is arbitrary?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ah this is to match up with the exact naming format that Bundler creates the directory with in vendor/cache for gems installed from Git

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oh, fun! probably worth a comment then, given this is load-bearing

@p0deje
Copy link
Member

p0deje commented Jan 29, 2026

Right now doesn't support Windows as I wasn't able to get this working with my limited Windows knowledge :(

What issues were you seeing? I wonder if those are related to git, and switching to the Bazel downloader would help?

@withered-magic
Copy link
Author

withered-magic commented Jan 29, 2026

Ah, it wasn't related to git - it was more that sync_bundle_cache.rb didn't seem to be working, i.e. for some reason Bundler wasn't able to install the gems from the synced vendor/cache any more. So not sure that switching to the Bazel downloader would be enough to fix, I can continue investigation though if want to fix that before landing this PR!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Support gems installed from Git repositories in rb_bundle_fetch(...)

3 participants