XML parsing and checksum mismatch errors at updates and checkouts

[Daniel Shahaf]

[ summary for dev@: bug in trunk that doesn't reproduce in 1.6.16;
I can reproduce it with trunk/ra_serf but not with trunk/ra_neon ]

I build Subversion as a part of TortoiseSVN (see
http://tortoisesvn.googlecode.com/svn/trunk/ext/build/subversion.build)
http://svn.webdav.org/repos/projects/neon/tags/0.29.5

I can reproduce it (with trunk svn):

* Checking out that URL over ra_neon proceeds normally.

* Checking out that URL over ra_serf is
(a) noticeably slower
(b) fails with a checksum error:

svn: E200014: Checksum mismatch for '/tmp/svn/neon-0.29.5-via-serf/src/ne_auth.c':
expected: 4ce0a5989ad4d1b0f7f66ae08c1e05c1
actual: 5848237f8c173bd25a118db45cd5574a

Also I reproduced this problem on our internal host with mod_dav_svn
version 1.6.5 (r38866).

Thanks for this. I'm not sure what causes this, but I've CCed dev@;
let's see what they have to say.

* Does this happen with neon, serf, or both?
* Can you build in debug mode (pass --debug to gen-make.py, if you're
using the build method from our INSTALL) to get some more information?
* Are any of those repositories public? (so that we can try accessing
them ourselves)
* What versions of mod_dav_svn do those servers run?
(You can check by navigating to the repository's URL in a Web browser)

Hi.
Sometimes at checkouts and updates via TortoiseSVN which built with
latest Subversion code
(http://svn.apache.org/repos/asf/subversion/trunk) I get "XML parsing
error (200:OK)" and "Checksum mismatch" errors. This happens on
different repositories (e.g. XML parsing error
could happen at Subversion code update).
This problem is not observed when I use TortoiseSVN with svn 1.6.* (e.g.
latest TortoiseSVN release).
--
Aleksandr Platonov

[Philip Martin]

Post by Daniel Shahaf
I can reproduce it with trunk/ra_serf but not with trunk/ra_neon ]

I build Subversion as a part of TortoiseSVN (see
http://tortoisesvn.googlecode.com/svn/trunk/ext/build/subversion.build)
http://svn.webdav.org/repos/projects/neon/tags/0.29.5

* Checking out that URL over ra_neon proceeds normally.
* Checking out that URL over ra_serf is
(a) noticeably slower
expected: 4ce0a5989ad4d1b0f7f66ae08c1e05c1
actual: 5848237f8c173bd25a118db45cd5574a

I was getting that error last month:

http://svn.haxx.se/dev/archive-2011-03/0745.shtml

What I find is that the problem sometimes appears and sometimes does
not. It's not predictable either, repeating a failed export/checkout
will sometimes work and sometimes fail at a different place. That is
the case when I try that neon URL:

svn: E200014: Checksum mismatch for 'subversion/tests/xx/src/ne_openssl.c':
expected: 3bd789b372ca14793e8ac228cdf43a15
actual: be72d4221d805da2484b1cd0182b9049

svn: E200014: Checksum mismatch for 'subversion/tests/xx/src/ne_ssl.h':
expected: 6fbb6c5dda02229623c887a9a08b1533
actual: 4f0c8e04845b70ebf1e88adbb780a2f0

I suspect the pool/memory handling in serf or libsvn_ra_serf, but I'm
not certain.

[Bert Huijben]

-----Original Message-----
Sent: vrijdag 8 april 2011 11:11
To: Daniel Shahaf
Subject: Re: XML parsing and checksum mismatch errors at updates and
checkouts

Post by Daniel Shahaf
I can reproduce it with trunk/ra_serf but not with trunk/ra_neon ]

I build Subversion as a part of TortoiseSVN (see

http://tortoisesvn.googlecode.com/svn/trunk/ext/build/subversion.build)

Post by Daniel Shahaf

http://svn.webdav.org/repos/projects/neon/tags/0.29.5

* Checking out that URL over ra_neon proceeds normally.
* Checking out that URL over ra_serf is
(a) noticeably slower
svn: E200014: Checksum mismatch for '/tmp/svn/neon-0.29.5-via-
expected: 4ce0a5989ad4d1b0f7f66ae08c1e05c1
actual: 5848237f8c173bd25a118db45cd5574a

http://svn.haxx.se/dev/archive-2011-03/0745.shtml
What I find is that the problem sometimes appears and sometimes does
not. It's not predictable either, repeating a failed export/checkout
will sometimes work and sometimes fail at a different place. That is
svn: E200014: Checksum mismatch for
expected: 3bd789b372ca14793e8ac228cdf43a15
actual: be72d4221d805da2484b1cd0182b9049
expected: 6fbb6c5dda02229623c887a9a08b1533
actual: 4f0c8e04845b70ebf1e88adbb780a2f0
I suspect the pool/memory handling in serf or libsvn_ra_serf, but I'm
not certain.

The update in one fetch operation in neon doesn't receive all checksums in
all places. Serf receives a few more, because it uses a request per file.
For trunk I fixed the missing checksums for 1.7, so a trunk server with a
trunk client gets all the checksums editor v1 supports.

But using an older server or an older client will not. In that case you just
get the calculated checksums in your working copy.

I think this server is pre 1.5, but it is certainly possible to get invalid
checksums in a checkout when you encounter the problems mentioned in issue
#3711. But you don't encounter these problems with trunk clients.

Bert

[Philip Martin]

Post by Bert Huijben

Post by Philip Martin
What I find is that the problem sometimes appears and sometimes does
not. It's not predictable either, repeating a failed export/checkout
will sometimes work and sometimes fail at a different place. That is
svn: E200014: Checksum mismatch for
expected: 3bd789b372ca14793e8ac228cdf43a15
actual: be72d4221d805da2484b1cd0182b9049
expected: 6fbb6c5dda02229623c887a9a08b1533
actual: 4f0c8e04845b70ebf1e88adbb780a2f0
I suspect the pool/memory handling in serf or libsvn_ra_serf, but I'm
not certain.

The update in one fetch operation in neon doesn't receive all checksums in
all places. Serf receives a few more, because it uses a request per file.
For trunk I fixed the missing checksums for 1.7, so a trunk server with a
trunk client gets all the checksums editor v1 supports.
But using an older server or an older client will not. In that case you just
get the calculated checksums in your working copy.
I think this server is pre 1.5, but it is certainly possible to get invalid
checksums in a checkout when you encounter the problems mentioned in issue
#3711. But you don't encounter these problems with trunk clients.

I was getting those checksum errors while doing an export, I'm not sure
how 3711 is relevant.

[Philip Martin]

Post by Philip Martin
I was getting those checksum errors while doing an export, I'm not sure
how 3711 is relevant.

So I got an checksum error during an export using serf for the file
src/ne_uri.c. I compared the "bad" file in the export with a "good"
version from a checkout using neon:

$ diff -a bad/src/svn-DfdRjG good/src/ne_uri.c
86,90c86,110
< #define URI_SEGCHAR (URI_PCHAR | FOT, OT, OT, OT, OT, OT, OT, OT, OT, O2010-10-14T15:02:41.455674Z *010-10-14T15:02:41.455674Z���
*DAV:getlastmodifiedThu, 14 Oct 2010 15:02:41 GMT�*Thu, 14 Oct 2010 15:02:41 GMTuN��p*�*DAV:version-name1823(+1823�+Z�+
+DAV:creator-displaynamejoe�+joe4r�p`+�+http://subversion.tigris.org/xmlns/dav/md5-checksum###error###,
###error###�<�p,3�(.@-http://subversion.tigris.org/xmlns/dav/�RIW�'(,Ơ��
,http://subversion.tigris.org/xmlns/dav/repository-uuid61a7d7f5-40b7-0310-9c16-bb0ea8cb1845�$61a7d7f5-40b7-0310-9c16-bb0ea8cb1845doH8�-�DAV:supportedlock
<
< X.
<
/T, OT, OT, OT, OT, OT, teK����.DAV:lockdiscovery/����
---

Post by Philip Martin
#define URI_SEGCHAR (URI_PCHAR | FS)
/* query = *( pchar / "/" / "?" ) */
#define URI_QUERY (URI_PCHAR | FS | QU)
/* fragment == query */
#define URI_FRAGMENT URI_QUERY
/* any characters which should be path-escaped: */
#define URI_ESCAPE ((URI_GENDELIM & ~(FS)) | URI_SUBDELIM | OT | PC)
static const unsigned int uri_chars[256] = {
/* 0xXX x0 x2 x4 x6 x8 xA xC xE */
/* 0x */ OT, OT, OT, OT, OT, OT, OT, OT, OT, OT, OT, OT, OT, OT, OT, OT,
/* 1x */ OT, OT, OT, OT, OT, OT, OT, OT, OT, OT, OT, OT, OT, OT, OT, OT,
/* 2x */ OT, SD, OT, GD, SD, PC, SD, SD, SD, SD, SD, PS, SD, DS, DT, FS,
/* 3x */ DG, DG, DG, DG, DG, DG, DG, DG, DG, DG, CL, SD, OT, SD, OT, QU,
/* 4x */ AT, AL, AL, AL, AL, AL, AL, AL, AL, AL, AL, AL, AL, AL, AL, AL,
/* 5x */ AL, AL, AL, AL, AL, AL, AL, AL, AL, AL, AL, GD, OT, GD, OT, US,
/* 6x */ OT, AL, AL, AL, AL, AL, AL, AL, AL, AL, AL, AL, AL, AL, AL, AL,
/* 7x */ AL, AL, AL, AL, AL, AL, AL, AL, AL, AL, AL, OT, OT, OT, TD, OT,
/* 8x */ OT, OT, OT, OT, OT, OT, OT, OT, OT, OT, OT, OT, OT, OT, OT, OT,
/* 9x */ OT, OT, OT, OT, OT, OT, OT, OT, OT, OT, OT, OT, OT, OT, OT, OT,
/* Ax */ OT, OT, OT, OT, OT, OT, OT, OT, OT, OT, OT, OT, OT, OT, OT, OT,
/* Bx */ OT, OT, OT, OT, OT, OT, OT, OT, OT, OT, OT, OT, OT, OT, OT, OT,
/* Cx */ OT, OT, OT, OT, OT, OT, OT, OT, OT, OT, OT, OT, OT, OT, OT, OT,
/* Dx */ OT, OT, OT, OT, OT, OT, OT, OT, OT, OT, OT, OT, OT, OT, OT, OT,