SVN::Web and remote repositories

nik on 2006-09-08T16:05:25

I've had a couple of journeys in to London over the last few weeks, and I've been able to use the time on the train to include support for remote Subversion repositories in to SVN::Web.

In use it's identical to 0.49, except that you can now specify the repositories to browse using Subversion URLs.

So if you have this config fragment in your config.yaml:

   repos:
     my-repo: /path/to/my/repo


that still works. But you can now do:

   repos:
     my-repo: file:///path/to/my/repo


That's no great change. But you can also do this:

   repos:
     niks-repo: svn://jc.ngo.org.uk/
     kde-repo: svn://anonsvn.kde.org/home/kde/


to link to remote repositories and browse them. So if your favourite open source project is using Subversion, but is using SomeOther(tm) web repository browser frontend you can fix that, without insisting that they install SVN::Web :-)

Obviously, this is going to be somewhat slower, so I recommend using the caching options in SVN::Web to speed things up. Also, it only seems to work for file:/// and svn:// URIs at the moment.

See this message http://jc.ngo.org.uk/pipermail/svnweb/2006-September/000018.html> for more details. The work's currently on a branch, retrievable with
svn checkout svn://jc.ngo.org.uk/nik/CPAN/SVN-Web/branches/svn_ra/

Feedback welcomed.


Bah

jk2addict on 2006-09-08T16:12:44

I feel your pain. I think the SVN API, and its bindings are quite a bit much less than stellar or intuitive.

Just for the sake of saying so, It is possible to browse using file, svn, AND http in http://search.cpan.org/user/claco/Catalyst-Model-SVN-0.05/

That's what powers http://handelframework.com/source/

It wasn't fun, and it wasn't pretty, but it does work. My biggest beef is the fetching content/logs on a tab/branch url. I can't just ask for it by url, since it's a copy; you have to stroll through the logs till copy event, and then get the original from its original location at the time of the copy.

Makes some sense, but it's certinaly not a way to do the API in a usable manner.

Re:Bah

nik on 2006-09-15T13:54:22

Just for the sake of saying so, It is possible to browse using file, svn, AND http in http://search.cpan.org/user/claco/Catalyst-Model-SVN-0.05/


Do you see much use of the http:/// access?

I've done some benchmarking comparing SVN::Repos (file:/// only) with SVN::Ra (file:/// and svn://). Carrying out 11 runs each of SVN::Web's test suite, SVN::Ra takes about 15% longer to run the tests than SVN::Repos.
x SVN::Repos
+ SVN::Ra
: = Mean
M = Median
+----------------------------------------------------------------------- --+
|                                     x                 +                  |
|                                     x                 +                  |
|                                     x                 +                  |
|                  x                  +                 +                  |
|                  x                  +                 +                 +|
|x                 x                  +                 +                 +|
|                |___________:________M___|                                |
|                                        |____________:_M__________|       |
+--------------------------------------------------------------------------+
    N           Min           Max        Median          Mean        Stddev
x  11             7             9             9     8.5454545    0.68755165
+  11             9            11            10     9.9090909    0.70064905
Difference at 99.5% confidence
        1.36364 +/- 1.05132
        15.9574% +/- 12.3026%
Using SVN::Client is even slower -- no hard figures yet, but I'm working on that.

I'm not sure that's an overhead that's worth paying. Have you got any evidence to support one or the other, anecdotal or otherwise?

Re:Bah

jk2addict on 2006-09-15T21:52:22

I've got nothing. I chose SVN::Client because it was the only one that had any decent pod. And I use Ra to get_latest_version from the repo since I couldn't find another way to do it using Client. And Ra totally baffles me to this day.

The SVN Perl bindings are not so readable. Trolling through C source docs/doxygen (when it works) isn't my idea of usable pod w/ examples. :-)

I was under the impression that Ra, or at least it's perl bindings couldn't do all forms of connections (svn, file, http). I'm sure http repo access in Model::SVN is slow...but I just wanted it to be an option. With some good caching code, it means I could reference 3rd party svn stuffs in a single site, or get a decent browse from a repo where no other methods are available due to firewall issues.

What was the question again? :-)

Re:Bah

nik on 2006-09-18T07:54:55

The SVN Perl bindings are not so readable. Trolling through C source docs/doxygen (when it works) isn't my idea of usable pod w/ examples. :-)

That has recently (like, last week) improved, as a big commit was made to Subversion to expand on the Perl documentation and provide more examples.

But yes, I've often had to fallback to writing a short hack to exercise a particular API function and use Data::Dumper on the result to divine what's being returned.

I was under the impression that Ra, or at least it's perl bindings couldn't do all forms of connections (svn, file, http).

SVN::Repos connects directly to the repository -- no URL scheme, you have to give it a path. So it's fastest.

SVN::Ra can do file:// and svn://. In my tests it fails trying to do http:/// -- I think this is because the SVN::Ra layer (as opposed to the underlying C API) always tries to fetch directory properties, which isn't always supported with http://./

It's connection orientied -- open a repository, perform many operations against it, close it. So it's slower than ::Repos (because of the intervening URL layer), but only 15% slower.

SVN::Client splits some of the ::Ra operations out in two different operations. Which is why it works against http:/// accessable repositories. But SVN::Client isn't connection oriented as far as I can tell -- every operation you carry out using its API means that, under the hood, it has to re-open the repo you're interested in, get the data, and disconnect.

I don't do that much in SVN::Web -- but I do do it when browsing a repository directory -- I have to make one call to SVN::Client::ls() to get the directory entries, and then I need to make n calls (where n is the number of files in the directory) to retrieve each file's MIME type, so that I can pass that info to the template, so the template can choose different icons to display for the files.

This is slow.

I might be able to work around it, but that needs more investigation.

I'm sure http repo access in Model::SVN is slow...but I just wanted it to be an option. With some good caching code, it means I could reference 3rd party svn stuffs in a single site, or get a decent browse from a repo where no other methods are available due to firewall issues.

Yeah, that's a nice feature. SVN::Web can do that now for svn:// accessible repos.

I'll keep banging away at SVN::Client and see if I can come up with something that works and is fast.

Very cool

Aristotle on 2006-09-09T08:21:02

This is great. It always bugged me when people would send me to a raw SVN repo URI without a useful web interface laid over it; no need to be annoyed about it any longer I guess. Thanks! :-)