Solve and understand HTTP Error 414: Request-URI Too Large

[Daniel Siepmann]

Hello everyone,

the error "HTTP Error 414: Request-URI Too Large" occurs regularly on our
project.
We develop new features in their own branches and some colleagues don't
close them.
If someone in our team doesn't pull often, he will get the error "HTTP
Error 414: Request-URI Too Large".

We raised the limits on our web server, but I want to understand why this
problem exists.
How does mercurial generate the URLs for a pull-Request? Why does this url
become longer and longer over time?
And is there a way to solve this problem instead of raising the limits on
server side?

Currently we just pull the important branches to "solve" the problem. But
that's just a workaround.

I'm looking forward your help and explanations.

Regards,

Daniel Siepmann.

[Mads Kiilerich]

Post by Daniel Siepmann
Hello everyone,
the error "HTTP Error 414: Request-URI Too Large" occurs regularly on
our project.
We develop new features in their own branches and some colleagues
don't close them.
If someone in our team doesn't pull often, he will get the error "HTTP
Error 414: Request-URI Too Large".
We raised the limits on our web server, but I want to understand why
this problem exists.
How does mercurial generate the URLs for a pull-Request? Why does this
url become longer and longer over time?
And is there a way to solve this problem instead of raising the limits
on server side?
Currently we just pull the important branches to "solve" the problem.
But that's just a workaround.

http://selenic.com/hg/rev/510cafe72004 is another workaround for the
bad use of Mercurial - it will make the request half as big.

/Mads

[Steve Barnes]

Post by Mads Kiilerich

Post by Daniel Siepmann
Hello everyone,
the error "HTTP Error 414: Request-URI Too Large" occurs regularly on
our project.
We develop new features in their own branches and some colleagues
don't close them.
If someone in our team doesn't pull often, he will get the error
"HTTP Error 414: Request-URI Too Large".
We raised the limits on our web server, but I want to understand why
this problem exists.
How does mercurial generate the URLs for a pull-Request? Why does
this url become longer and longer over time?
And is there a way to solve this problem instead of raising the
limits on server side?
Currently we just pull the important branches to "solve" the problem.
But that's just a workaround.

http://selenic.com/hg/rev/510cafe72004 is another workaround for the
bad use of Mercurial - it will make the request half as big.
/Mads
_______________________________________________
Mercurial mailing list
http://selenic.com/mailman/listinfo/mercurial

The other workaround - assuming that people do actually commit & push
reasonable regularly - even if some people don't pull as often as they
should do - hg log -r -1 to find out your current revision, say it is
250 then use hg pull -r 251:300, etc., - if that hits the problem then
try -r 251:275 - so as to pull the changes in manageable chunks.

Also remind people that if the did regular pulls they wouldn't get the
problem.

BTW I did have the same problem with a hg clone of an SVN repo where
someone had zipped up a 40 GB binary toolchain and committed it -
twice! The workaround there was to pull skipping those two commits and
the solution was to get the SVN admins to remove the illegal commits.

Gadget/Steve

[Matt Mackall]

Post by Daniel Siepmann
Hello everyone,
the error "HTTP Error 414: Request-URI Too Large" occurs regularly on our
project.
We develop new features in their own branches and some colleagues don't
close them.
If someone in our team doesn't pull often, he will get the error "HTTP
Error 414: Request-URI Too Large".
We raised the limits on our web server, but I want to understand why this
problem exists.
How does mercurial generate the URLs for a pull-Request? Why does this url
become longer and longer over time?
And is there a way to solve this problem instead of raising the limits on
server side?
Currently we just pull the important branches to "solve" the problem. But
that's just a workaround.
I'm looking forward your help and explanations.

Let's start with the single most important piece of information:

What version of Mercurial are you using?

[Daniel Siepmann]

Our Version on the Server is 2.2.2.
On client side it depends on the local machine. In my own case it's 3.0.1
but there are still older versions in use.

I tried some sniffer tools to get the generated URL and understand what's
causing the long URL. But I couldn't fetch the URL ...
So what's the Problem? The heads, Branches, even closed branches?

Post by Matt Mackall

Post by Daniel Siepmann
Hello everyone,
the error "HTTP Error 414: Request-URI Too Large" occurs regularly on our
project.
We develop new features in their own branches and some colleagues don't
close them.
If someone in our team doesn't pull often, he will get the error "HTTP
Error 414: Request-URI Too Large".
We raised the limits on our web server, but I want to understand why this
problem exists.
How does mercurial generate the URLs for a pull-Request? Why does this

url

Post by Daniel Siepmann
become longer and longer over time?
And is there a way to solve this problem instead of raising the limits on
server side?
Currently we just pull the important branches to "solve" the problem. But
that's just a workaround.
I'm looking forward your help and explanations.

What version of Mercurial are you using?
--
Mathematics is the supreme nostalgia of our time.

[Matt Mackall]

Post by Daniel Siepmann
Our Version on the Server is 2.2.2.
On client side it depends on the local machine. In my own case it's 3.0.1
but there are still older versions in use.
I tried some sniffer tools to get the generated URL and understand what's
causing the long URL.

Can you send us the sniffed URL _and_ headers?

Which webserver are you using?

Post by Daniel Siepmann
But I couldn't fetch the URL ...
So what's the Problem? The heads, Branches, even closed branches?

Typically, it's going to be "topological heads" as shown by "hg heads
-t".

[Daniel Siepmann]

Hey Matt,

we are using Apache/2.2.16 (Debian).
I could not fetch the url and headers using a sniffer tool. I just got
Hexvalues.
A teammate also thinks the url is build using all topological heads.
Currently some mates are closing their branches AFTER merging them. This
way they are still a head that needs to be pulled.
We are merging them again and defined our workflow to close branches and
merge their heads afterwards.
We hope to solve the problem this way. Thanks for your help. I'll post our
experience with the changed workflow here to finish the thread.

Post by Matt Mackall

Post by Daniel Siepmann
Our Version on the Server is 2.2.2.
On client side it depends on the local machine. In my own case it's 3.0.1
but there are still older versions in use.
I tried some sniffer tools to get the generated URL and understand what's
causing the long URL.

Can you send us the sniffed URL _and_ headers?
Which webserver are you using?

Post by Daniel Siepmann
But I couldn't fetch the URL ...
So what's the Problem? The heads, Branches, even closed branches?

Typically, it's going to be "topological heads" as shown by "hg heads
-t".
--
Mathematics is the supreme nostalgia of our time.

[Pierre-Yves David]

Post by Daniel Siepmann
Hey Matt,
we are using Apache/2.2.16 (Debian).
I could not fetch the url and headers using a sniffer tool. I just got
Hexvalues.
A teammate also thinks the url is build using all topological heads.
Currently some mates are closing their branches AFTER merging them. This
way they are still a head that needs to be pulled.
We are merging them again and defined our workflow to close branches and
merge their heads afterwards.
We hope to solve the problem this way. Thanks for your help. I'll post
our experience with the changed workflow here to finish the thread.

Since quite a while (Mercurial 1.8 or 1.9 if I remember correctly),
Mercurial is using a way to exchange the heads that is behind smarter
about the way it communicate heads. So there is probably some old
clients or server involved here. If not, you have found some interesting
bugs.

[Daniel Siepmann]

Hello everyone,

we got the issue and figured a way to work with Mercurial to prevent the
issue. We just didn't understand the way Mercurial works in this area.
So to finish this mail and deliver the results to the world, here it is,
hopefully helpful for others too:

Mercurial create topological heads. This are heads without any children.
They don't depend on other kinds like branches.
While processing a "pull", Mercurial will check each topological head for
new children. So by a growing number of topological heads the url will grow
too.

All you need to do is to resolve this topological heads. Just to close a
branch doesn't delete the topological head as you always can reopen it and
Mercurial, still needs to check for children.
Said that, you need to "remove" the heads by using a merge with another
branch. This way you will remove one head.
If this head was a "dead end", so something you don't want to have in the
merge, just remove the changes during the merge, i.e. "use theirs".

In our case we didn't know the internal work with topological heads, now we
don't have anymore problems.

Regards, Daniel.

On Tue, Sep 2, 2014 at 12:41 AM, Pierre-Yves David <

Post by Pierre-Yves David

Post by Daniel Siepmann
Hey Matt,
we are using Apache/2.2.16 (Debian).
I could not fetch the url and headers using a sniffer tool. I just got
Hexvalues.
A teammate also thinks the url is build using all topological heads.
Currently some mates are closing their branches AFTER merging them. This
way they are still a head that needs to be pulled.
We are merging them again and defined our workflow to close branches and
merge their heads afterwards.
We hope to solve the problem this way. Thanks for your help. I'll post
our experience with the changed workflow here to finish the thread.

Since quite a while (Mercurial 1.8 or 1.9 if I remember correctly),
Mercurial is using a way to exchange the heads that is behind smarter about
the way it communicate heads. So there is probably some old clients or
server involved here. If not, you have found some interesting bugs.
--
Pierre-Yves David

[Pierre-Yves David]

Post by Daniel Siepmann
Hello everyone,
we got the issue and figured a way to work with Mercurial to prevent the
issue. We just didn't understand the way Mercurial works in this area.
So to finish this mail and deliver the results to the world, here it is,
Mercurial create topological heads. This are heads without any children.
They don't depend on other kinds like branches.
While processing a "pull", Mercurial will check each topological head
for new children. So by a growing number of topological heads the url
will grow too.
All you need to do is to resolve this topological heads. Just to close a
branch doesn't delete the topological head as you always can reopen it
and Mercurial, still needs to check for children.
Said that, you need to "remove" the heads by using a merge with another
branch. This way you will remove one head.
If this head was a "dead end", so something you don't want to have in
the merge, just remove the changes during the merge, i.e. "use theirs".
In our case we didn't know the internal work with topological heads, now
we don't have anymore problems.
Regards, Daniel.

This is probably related to this Mercurial bugs:

http://bz.selenic.com/show_bug.cgi?id=4411