Bitbucket ends support for Mercurial (Hg), a quick guide

I have been a long-time user of Bitbucket for my personal projects, over 10 years now, I believe. My preferred source control system has always been Mercurial (Hg), especially in those early days, when I found the tools for Git on Windows to be quite unstable and plagued with compatibility issues. Using TortoiseHg on Windows was very straightforward and reliable.

As a result, all of my repositories that I have created on Bitbucket over the years, have been Mercurial ones. However, Git appears to have won the battle in the end, and this has triggered Bitbucket to stop supporting Mercurial repositories. They will no longer allow you to create new Mercurial repositories starting February 1st 2020, and by June 2020, they will shut down Mercurial access altogether, and what’s worse: they will *delete* all your existing Mercurial repositories.

So basically you *have* to migrate your Mercurial repositories before June 1st 2020, or else you lose your code and history forever.

Now, given such a decision, it would have been nice if Bitbucket had offered an automatic migration service, but alas, there is no such thing. You need to manually convert your repositories. I suppose the most obvious choice is to migrate them to Git repositories on Bitbucket. That is what I have done. There are various ways to do it, and various sources that give you half-baked solutions using half-baked tools. So I thought I’d explain how I did it, and point out the issues I ran into.

Tools

First of all, we have to choose the tools we want to use for this migration. I believe the best tool for the job is the hg-git Mercurial plugin. It allows the hg commandline tool to access Git repositories, which means you can push your existing Hg repository directly to Git.

As I said, I use TortoiseHg, and they already include the hg-git plugin. You need to enable it though, by ticking its checkbox. Go to File->Settings, and enable it in the dialog:

hg-git

Sadly, that turned out not to work very well. There is a problem with the distributed plugins, which causes the hg-git extension to crash with a strange ‘No module named selectors!’ message. This issue here discusses it, although there is no official fix yet. But if you scroll down, you do find a zip file with a fixed distribution of the hg-git plugin and its dependencies. Download that file, unzip it into the TortoiseHg folder (replacing the existing contents in the lib folder), and hg-git is ready for action!

On the Git-side, I use TortoiseGit. Git is a bit ‘special’ though. That is, most Git-tools do not include Git itself, but expect that you have a binary Git distribution (git.exe and supporting tools/libs) installed already. For Windows, there is Git for Windows to solve that dependency. Install that first, and then install TortoiseGit (or whatever tool you want to use. Or just use Git directly from the commandline). But first I want to mention an important ‘snag’ with Git on Windows (or any platform for that matter).

Git wants to convert line-endings when committing or checking out code. On UNIX-like systems, you generally don’t notice, because Git wants to default to having LF as the line-ending for all text-files in a repository. LF happens to be the default on UNIX-like systems, so effectively there is not usually any conversion going on. On Windows, CRLF is the default line-ending, so that would mean that text files on a Windows system are converted to LF when you commit them, and converted back to CRLF when you check out.

Git argues that this is better for cross-platform compatibility. Personally I think this is a bad idea. There are two reasons why:

  1. Just because you are using Git from a Windows system does not necessarily mean you are using CRLF line endings (you may be checking out code for a different platform, or you are using tools that do not use CRLF). Most Windows software is quite resilient to both types of line-endings, and various tools use LF endings even on Windows (like Doxygen for example, it generates HTML files with LF endings. Which is not a problem, because browsers can handle HTML with either line-ending). Some tools even require LF endings, else they do not work.
  2. Git cannot reliably detect whether a file is text or binary. This means that you have to add exceptions to a .gitattributes file to take care of any special cases. Which you usually find AFTER you’ve committed them, and they turn out to break something when someone else checks them out.

So I would personally suggest to not use the automatic conversion of line-endings. I prefer committing and checking out as-is. Just make sure you get the file right on the initial commit, and it will check out correctly on all systems.

When you install Git for Windows, make sure you select as-is on the following dialog during installation:

install-[1]-git-03

As it says, it is the core.autocrlf setting, in case you want to change it later.

I would argue that the whole CRLF/LF thing is a non-issue anyway. Most version control systems do not perform these conversions. In fact, oftentimes code is distributed as a tarball or a zip file. When you extract those, you don’t get any conversion either. But still you can compile that same code on various platforms without doing any conversion at all.

Using hg-git

The process I used to convert each repository is a simple 7-step process. I based it on the article you can find here, but modified it somewhat for this specific use-case.

1) Rename your existing repository in Bitbucket. I do this by adding ‘Hg’ to the end. So for example if I have “My Repository”, I change its name to “My Repository Hg”. This means that the URL will also change, so you can no longer accidentally clone from or commit to this repository from any working directories/tools. It also means that you can create the new Git repository under the same name as your original Hg repository.

2) Do a clean hg clone into a new directory. This makes sure you don’t run into any problems with your working directory having uncommitted changes, or perhaps a corrupted local history or such. You can just use TortoiseHg to create this new clone, or run hg from the commandline.

3) Create the new Git repository on Bitbucket, and grab its URL (eg https://(username)@bitbucket.org/(username)/my-repository.git)

Note that you can use either https or ssh access for Git. However, I have found it to be quite troublesome to get ssh set up and working under Windows, so I would recommend using https here.

4) Open a command prompt, go to the directory of your clean hg clone from step 2, and run the following command:

hg bookmark -r default master

This command is very important, because it links the ‘default’ branch of your Hg repository to the ‘master’ branch of your Git repository. In both environments they have a special status.

5) Now push your local Hg repository to the new Git repository using hg-git:

hg push https://(username)@bitbucket.org/(username)/my-repository.git

At this point your new Git repository should be filled with a complete copy of your Hg repository, including all the history. You can now delete this local clone. If you have any working directories you want to change to Git, the final 3 steps will explain how to do that.

(Note: I am not entirely sure, but I believe hg-git will always do an as-is push to Git, so no changing of line-endings. I don’t think it even uses git.exe at all, so it probably will not respond to the Git for Windows configuration anyway. At any rate, it did an as-is push with my repositories, and I could not find any setting or documentation on hg-git for changing line-endings).

6) Analogous to step 2, perform a new git clone from the repository into a new directory, to make sure there can be no existing files or other repository state that may corrupt things.

7) Make sure your working directory is at the latest commit of the default branch of your Hg repository (you may need to modify the .hgrc file, because it will still point to the old URL of the Hg repository, and we have renamed it in step 1, so you need to set it to the correct URL). Since this step and the next one are somewhat risky, you might want to create a copy of your working directory first, so you can always go back to Hg if the transition to Git didn’t go right.

8) Remove the (hidden) .hg directory from the existing working directory of your old Hg repository, and copy the (hidden) .git directory from the directory in step 6. This will effectively unlink your working directory from Hg altogether, and link it to Git (also at the latest commit).

You can now delete the local clone from step 6.

Note that if you chose a different line-ending option than ‘as-is’, you may find that Git now shows a lot of changes in your files. And when you do a diff, you don’t see any changes. That’s because Git considers it a change when it has to modify the line-endings, but diff does not show line-endings as changes.

This entry was posted in Software development and tagged , , , , , , , , , . Bookmark the permalink.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s