Is it possible to synchronize local/remote repositories without any interaction (i.e. by script)?

[Chris Green]

I am a 'single user' developer (retired Software Engineer) and I use
mercurial to keep track of software I write mostly for my own use.
It's an excellent tool, much more intuitive than git for me anyway.


I would like to be able to keep some repositories on remote systems
in step with a 'master' repository on my home desktop machine. This
is all on Linux (Debian or derivatives) systems. All the systems have
ssh access to each other but in the main this requires a password or
key/passphrase for authentication.


Firstly, is there a straightforward sequence of commands that one can
issue to 'synchronize' two repositories? Since I am almost certainly
the only user and I'm unlikely to edit the same file on two different
systems (without committing changes in between) no file merges should
be required. I'm happy for a script to ask for input if it does
detect that some sort of merge is required.

So, I've cloned a repository on a remote system somewhere:-

hg clone ssh://my.home.system.com/repo1

I have two use cases:-

1 - I've changed something on the home system and I want the
change to be copied to the remote system by a script called from
.bash_login (i.e. automatically at login).

2 - After logging in on the remote system via ssh I change
something there and I want the change copied back to the home
system by the .bash_logout script (i.e. automatically at logout).

These need to be as far as possible silent scripts, as noted above I'm
happy to have them talk to me if there's a conflict of some sort but
otherwise I want quietness. I can probably manage to make the ssh
access password/key-less but how easy is it to automate the mercurial
bits?

[Adrian Klaver]

Post by Chris Green
I am a 'single user' developer (retired Software Engineer) and I use
mercurial to keep track of software I write mostly for my own use.
It's an excellent tool, much more intuitive than git for me anyway.
I would like to be able to keep some repositories on remote systems
in step with a 'master' repository on my home desktop machine. This
is all on Linux (Debian or derivatives) systems. All the systems have
ssh access to each other but in the main this requires a password or
key/passphrase for authentication.
Firstly, is there a straightforward sequence of commands that one can
issue to 'synchronize' two repositories? Since I am almost certainly
the only user and I'm unlikely to edit the same file on two different
systems (without committing changes in between) no file merges should
be required. I'm happy for a script to ask for input if it does
detect that some sort of merge is required.

The way I did/do this:

1) Setup a private repo at Bitbucket(bitbucket.org), its free.

2) Use hg push and pull commands to move commits from local computer to
Bitbucket and then to remote computer.


I would say doing an automated script on login will cause you more
issues then it is worth. You will forget to commit after a session and
then the script will choke. While it is possible to code for the common
possible errors I find the feedback from running the commands manually
important.

Post by Chris Green
So, I've cloned a repository on a remote system somewhere:-
hg clone ssh://my.home.system.com/repo1
I have two use cases:-
1 - I've changed something on the home system and I want the
change to be copied to the remote system by a script called from
.bash_login (i.e. automatically at login).
2 - After logging in on the remote system via ssh I change
something there and I want the change copied back to the home
system by the .bash_logout script (i.e. automatically at logout).
These need to be as far as possible silent scripts, as noted above I'm
happy to have them talk to me if there's a conflict of some sort but
otherwise I want quietness. I can probably manage to make the ssh
access password/key-less but how easy is it to automate the mercurial
bits?

[Augie Fackler]

Post by Chris Green
These need to be as far as possible silent scripts, as noted above I'm
happy to have them talk to me if there's a conflict of some sort but
otherwise I want quietness. I can probably manage to make the ssh
access password/key-less but how easy is it to automate the mercurial
bits?

Should be pretty easy - `hg push && hg pull` is probably all you need, and if either of those exit nonzero you need to do some kind of merge.

[Chris Green]

Post by Augie Fackler

Post by Chris Green
These need to be as far as possible silent scripts, as noted above I'm
happy to have them talk to me if there's a conflict of some sort but
otherwise I want quietness. I can probably manage to make the ssh
access password/key-less but how easy is it to automate the mercurial
bits?

Should be pretty easy - `hg push && hg pull` is probably all you need,
and if either of those exit nonzero you need to do some kind of merge.

So if I make a change on a 'remote' machine and commit it to the local
repository then all I need do is 'push <central repository>' and the
change will be there too? I thought doing that would create a branch
and that someone/somewhere then had to do a merge on the 'central'
repository.

[Adrian Klaver]

Post by Chris Green

Post by Augie Fackler

Post by Chris Green
These need to be as far as possible silent scripts, as noted above I'm
happy to have them talk to me if there's a conflict of some sort but
otherwise I want quietness. I can probably manage to make the ssh
access password/key-less but how easy is it to automate the mercurial
bits?

Should be pretty easy - `hg push && hg pull` is probably all you need,
and if either of those exit nonzero you need to do some kind of merge.

So if I make a change on a 'remote' machine and commit it to the local
repository then all I need do is 'push <central repository>' and the
change will be there too? I thought doing that would create a branch
and that someone/somewhere then had to do a merge on the 'central'
repository.

hg help push

" Push changesets from the local repository to the specified destination.

This operation is symmetrical to pull: it is identical to a pull in
the destination repository from the current one.

By default, push will not allow creation of new heads at the
destination, since multiple heads would make it unclear which head to
use. In this situation, it is recommended to pull and merge before pushing.


...


Use --new-branch if you want to allow push to create a new named branch
that is not present at the destination. This allows you to only
create a new branch without forcing other changes.
"

So merges and branches are not automatically part of push/pull.

[Chris Green]

-----Original Message-----
From: Chris Green
Sent: Sunday, August 12, 2018 1:56 PM

Post by Augie Fackler

Post by Chris Green
These need to be as far as possible silent scripts, as noted
above I'm happy to have them talk to me if there's a conflict
of some sort but otherwise I want quietness. I can probably
manage to make the ssh access password/key-less but how easy is
it to automate the mercurial bits?

Should be pretty easy - `hg push && hg pull` is probably all you
need, and if either of those exit nonzero you need to do some kind
of merge.

So if I make a change on a 'remote' machine and commit it to the local
repository then all I need do is 'push <central repository>' and the
change will be there too? I thought doing that would create a branch
and that someone/somewhere then had to do a merge on the 'central'
repository.

Branching only happens when there are two or more commits with the same
parent. Since you are the only one committing to these repositories, as
long as all existing commits are in the central repository to be pulled
and you remember to update your working directory to tip after pulling
them and before you commit any new changes, then you aren't creating anonymous
branches.
I found the diagrams at http://hginit.com/02.html helpful when I was first learning.

OK, thanks all, I guess I'll just have to try it and see how things go.

[Cameron Simpson]

Chris,

I'm a little late here, but I'm in a similar situation to you. Here's what I
do.

1: I've got a personal repo at bitbucket:

https://bitbucket.org/cameron_simpson/css/overview

which I push to with:

hg push --new-branch -r tip ssh://***@bitbucket.org/cameron_simpson/css

which is normally silent (well, noninteractive - it reports on how many
changesets got pushed).

2: I live on my laptop, but I've got working repos on a couple of home
machines. Since the .hg/hgrc in the local repo has this:

[paths]
default = ssh://***@bitbucket.org/cameron_simpson/css
home = ssh://home/hg/css

I can just "hg push home" to update the home machine.

Now, you could stick any of these in an hg commit hook, and even kick them off
in the background with output redirected from there (to not stall your commits
and not produce annoying noise) because if you're offline or there's some other
failure/conflict you can always address it later.

You do want to keep in your head the difference between a repo and a checkout.
In my mind, the repo is the state (contained in the .hg subdir) and the
checkout is your working set of files. The "hg push" above only updates the
repo. The checked out files at the far end are _not_ updated.

For Bitbucket that is normal and irrelevant - it _only_ manages/provides a
repo. For the home machine it means the work tree has all the new state in the
repo but that the working files are not up to date.

Next time I stand in the home machine checkout I need to go "hg up" to bring
the working files up to date. You could automate or not. You _do_ want to do it
before doing new work as otherwise your next commit there will make a new
branch.

Cheers,
Cameron Simpson <***@cskk.id.au>

[Chris Green]

Post by Cameron Simpson
Chris,
I'm a little late here, but I'm in a similar situation to you. Here's what I
do.
https://bitbucket.org/cameron_simpson/css/overview
which is normally silent (well, noninteractive - it reports on how many
changesets got pushed).
2: I live on my laptop, but I've got working repos on a couple of home
[paths]
home = ssh://home/hg/css
I can just "hg push home" to update the home machine.
Now, you could stick any of these in an hg commit hook, and even kick them
off in the background with output redirected from there (to not stall your
commits and not produce annoying noise) because if you're offline or there's
some other failure/conflict you can always address it later.
You do want to keep in your head the difference between a repo and a
checkout. In my mind, the repo is the state (contained in the .hg subdir)
and the checkout is your working set of files. The "hg push" above only
updates the repo. The checked out files at the far end are _not_ updated.
For Bitbucket that is normal and irrelevant - it _only_ manages/provides a
repo. For the home machine it means the work tree has all the new state in
the repo but that the working files are not up to date.
Next time I stand in the home machine checkout I need to go "hg up" to bring
the working files up to date. You could automate or not. You _do_ want to do
it before doing new work as otherwise your next commit there will make a new
branch.

Thanks for the excellent explanation and clarification Cameron, that
is really useful.

[Bryan Murdock]

You can just use rsync or get fancy and use something like syncthing. I've
done both with mercurial repos and it works great. Your history *and*
working copy stay synced, you never have to commit just for the purpose of
synchronizing files.

Bryan

Post by Chris Green
I am a 'single user' developer (retired Software Engineer) and I use
mercurial to keep track of software I write mostly for my own use.
It's an excellent tool, much more intuitive than git for me anyway.
I would like to be able to keep some repositories on remote systems
in step with a 'master' repository on my home desktop machine. This
is all on Linux (Debian or derivatives) systems. All the systems have
ssh access to each other but in the main this requires a password or
key/passphrase for authentication.
Firstly, is there a straightforward sequence of commands that one can
issue to 'synchronize' two repositories? Since I am almost certainly
the only user and I'm unlikely to edit the same file on two different
systems (without committing changes in between) no file merges should
be required. I'm happy for a script to ask for input if it does
detect that some sort of merge is required.
So, I've cloned a repository on a remote system somewhere:-
hg clone ssh://my.home.system.com/repo1
I have two use cases:-
1 - I've changed something on the home system and I want the
change to be copied to the remote system by a script called from
.bash_login (i.e. automatically at login).
2 - After logging in on the remote system via ssh I change
something there and I want the change copied back to the home
system by the .bash_logout script (i.e. automatically at logout).
These need to be as far as possible silent scripts, as noted above I'm
happy to have them talk to me if there's a conflict of some sort but
otherwise I want quietness. I can probably manage to make the ssh
access password/key-less but how easy is it to automate the mercurial
bits?
--
Chris Green
_______________________________________________
Mercurial mailing list
https://www.mercurial-scm.org/mailman/listinfo/mercurial

[Cameron Simpson]

Post by Bryan Murdock
You can just use rsync or get fancy and use something like syncthing. I've
done both with mercurial repos and it works great. Your history *and*
working copy stay synced, you never have to commit just for the purpose of
synchronizing files.

Provided the far end never does any commits of its own...

Cheers,
Cameron Simpson <***@cskk.id.au>

[Chris Green]

Post by Cameron Simpson

Post by Bryan Murdock
You can just use rsync or get fancy and use something like syncthing. I've
done both with mercurial repos and it works great. Your history *and*
working copy stay synced, you never have to commit just for the purpose of
synchronizing files.

Provided the far end never does any commits of its own...

If using syncthing you can change things anywhere in any order
treating the hg repository as if it was on a shared disk drive.

The only problem might be with multiple users changing the same thing
in two places at the same time.

I find in practice that using syncthing works very well with me as the
only user, I just change things wherever I happen to be logged in at
the time and can commit from anywhere too. It's very handy for files
that are common/shared between multiple systems.

[Chris Green]

Post by Bryan Murdock
You can just use rsync or get fancy and use something like syncthing.

I am using syncthing and it works very well, it's just that I have
some small[er] machines where I wondered whether there might be better
ways to handle this,

I've actually decided to go with syncthing, it seems to run quite
happily fo a small collection of files on my Raspberry Pi (not the
latest).