jelmer/python-fastimport.git
14 years agoMerge Python 2.5 fix for darcs-fast-export
Ian Clatworthy [Mon, 25 Jan 2010 01:59:10 +0000 (11:59 +1000)]
Merge Python 2.5 fix for darcs-fast-export

14 years agodarcs-fast-export: don't use urllib.urlopen().getcode()
Miklos Vajna [Sun, 24 Jan 2010 23:45:31 +0000 (00:45 +0100)]
darcs-fast-export: don't use urllib.urlopen().getcode()

As it's missing from python2.5.

14 years agoInclude the known_graph code.
John Arbash Meinel [Tue, 5 Jan 2010 07:24:14 +0000 (01:24 -0600)]
Include the known_graph code.

14 years agoSwitch to using CommitBuilder
John Arbash Meinel [Fri, 11 Dec 2009 14:55:04 +0000 (08:55 -0600)]
Switch to using CommitBuilder
Rather than holding most of the logic ourselves.

14 years agoAdd a bunch of direct tests for the _TreeShim interface.
John Arbash Meinel [Wed, 9 Dec 2009 20:14:46 +0000 (14:14 -0600)]
Add a bunch of direct tests for the _TreeShim interface.

14 years agoMerge John's test refactoring - uses multiplication now
Ian Clatworthy [Tue, 8 Dec 2009 06:31:53 +0000 (16:31 +1000)]
Merge John's test refactoring - uses multiplication now

14 years agoMerge John's improvements to info_processor.py
Ian Clatworthy [Tue, 8 Dec 2009 06:29:47 +0000 (16:29 +1000)]
Merge John's improvements to info_processor.py

14 years agoMerge John's smarter caching of blobs to improve memory footprint
Ian Clatworthy [Tue, 8 Dec 2009 06:26:34 +0000 (16:26 +1000)]
Merge John's smarter caching of blobs to improve memory footprint

14 years agoFaster post-processing of the inventory delta
Ian Clatworthy [Fri, 4 Dec 2009 06:52:54 +0000 (16:52 +1000)]
Faster post-processing of the inventory delta

14 years agoSome debugging code. It looks like the main bugs involve files that are deleted and...
John Arbash Meinel [Tue, 1 Dec 2009 15:15:19 +0000 (09:15 -0600)]
Some debugging code. It looks like the main bugs involve files that are deleted and restored.

14 years agoWhen post-processing the delta stream, don't ask to generate a full inventory to...
John Arbash Meinel [Mon, 30 Nov 2009 21:31:31 +0000 (15:31 -0600)]
When post-processing the delta stream, don't ask to generate a full inventory to check for deletions.
Instead, just use 'create_by_apply_delta'.

14 years agoUse the new KnownGraph.add_node() functionality.
John Arbash Meinel [Mon, 30 Nov 2009 19:00:44 +0000 (13:00 -0600)]
Use the new KnownGraph.add_node() functionality.

This changes the file_id heads lookups to use a KnownGraph over all revision-ids
which should be a lot better than using a regular Graph instance.
Initial testing w/ xserver doesn't show much, since it seems to be purely
linear for quite a bit of x history.

Oddly enough the heads result seems to give a different number of file texts
which is worrying.

14 years agoThe fix that landed in bzr did not set builder.new_inventory. :(
John Arbash Meinel [Mon, 30 Nov 2009 17:57:02 +0000 (11:57 -0600)]
The fix that landed in bzr did not set builder.new_inventory. :(

This still works, but it means that we don't keep the in-memory deserialized bits
from the previous inventory. Instead we start from-scratch each time.

14 years agoMerge Jelmer's fix for exporters installation
Ian Clatworthy [Sun, 29 Nov 2009 10:37:32 +0000 (20:37 +1000)]
Merge Jelmer's fix for exporters installation

14 years agoFix installation of the exporters module.
Jelmer Vernooij [Thu, 26 Nov 2009 19:03:48 +0000 (20:03 +0100)]
Fix installation of the exporters module.

14 years agoDefault branch nick to mapped git ref name (Max Bowsher)
Ian Clatworthy [Mon, 23 Nov 2009 07:06:23 +0000 (17:06 +1000)]
Default branch nick to mapped git ref name (Max Bowsher)

14 years agobzr *does* run atexit functions when exiting, but doesn't run deconstructors.
John Arbash Meinel [Sun, 15 Nov 2009 20:42:36 +0000 (14:42 -0600)]
bzr *does* run atexit functions when exiting, but doesn't run deconstructors.

Also, shrink the 'small blob' size a bit to allow data to be reclaimed. Though it did
show up as *lots* of small files in the qt import. Something like 1-2k files in the
first 2 dumps.

14 years agoSwitch to closing the large-content blobs that we store to disk.
John Arbash Meinel [Fri, 13 Nov 2009 19:16:20 +0000 (13:16 -0600)]
Switch to closing the large-content blobs that we store to disk.

During the qt import, we end up with >2000 large blobs, and a few hundred MB of
small blobs. We don't want to create more small blobs, because their disk space
is not reclaimed, but >2000 is too many open file handles (at least on Windows).
Using filenames works, but we aren't as guaranteed that things will clean up
nicely.

14 years agoDump sticky blobs to disk when memory pressure gets high.
John Arbash Meinel [Fri, 13 Nov 2009 08:00:16 +0000 (02:00 -0600)]
Dump sticky blobs to disk when memory pressure gets high.

14 years agoFall back to the repository for cases where the content is not present in the stream...
John Arbash Meinel [Thu, 12 Nov 2009 23:25:04 +0000 (17:25 -0600)]
Fall back to the repository for cases where the content is not present in the stream yet.

14 years agoWe need to handle when the object has been deleted.
John Arbash Meinel [Thu, 12 Nov 2009 22:29:42 +0000 (16:29 -0600)]
We need to handle when the object has been deleted.

14 years agoFound a bug in CommitBuilder.finish_inventory().
John Arbash Meinel [Thu, 12 Nov 2009 22:23:23 +0000 (16:23 -0600)]
Found a bug in CommitBuilder.finish_inventory().

It turns out that the add_revision_by_delta code was not properly setting
inv_sha1, but it didn't matter because apparently the other code wasn't really
using it....

Anyway, this gives us a proper new inventory, and seems to generally be working.
The next step is that the 'merge' file text is not present because the
delta stream does not contain it...

14 years ago(broken) Start working towards using CommitBuilder rather than using a custom impleme...
John Arbash Meinel [Thu, 12 Nov 2009 14:07:24 +0000 (08:07 -0600)]
(broken) Start working towards using CommitBuilder rather than using a custom implementation.

14 years agoadd the failing test that we preserve the last-modified revision
John Arbash Meinel [Thu, 12 Nov 2009 08:02:42 +0000 (02:02 -0600)]
add the failing test that we preserve the last-modified revision

14 years agoChange to multiplying tests rather than manually.
John Arbash Meinel [Thu, 12 Nov 2009 07:51:21 +0000 (01:51 -0600)]
Change to multiplying tests rather than manually.

14 years agoChange (.keys(), .values()) to using (.iteritems())
John Arbash Meinel [Thu, 12 Nov 2009 05:37:43 +0000 (23:37 -0600)]
Change (.keys(), .values()) to using (.iteritems())

14 years agoMerge Max's fixes for tags created via commit commands
Ian Clatworthy [Fri, 6 Nov 2009 08:06:27 +0000 (18:06 +1000)]
Merge Max's fixes for tags created via commit commands

14 years agoAlso catch tagging via commit when resuming a crashed import.
Max Bowsher [Wed, 4 Nov 2009 01:58:36 +0000 (01:58 +0000)]
Also catch tagging via commit when resuming a crashed import.

14 years agoDefault branch-nick to mapped git ref name.
Max Bowsher [Wed, 4 Nov 2009 01:52:34 +0000 (01:52 +0000)]
Default branch-nick to mapped git ref name.

14 years agoStore the BranchMapper in the CacheManager so it can be got from other places.
Max Bowsher [Wed, 4 Nov 2009 01:27:59 +0000 (01:27 +0000)]
Store the BranchMapper in the CacheManager so it can be got from other places.

14 years agoMake BranchMapper just map one name per call.
Max Bowsher [Wed, 4 Nov 2009 01:04:34 +0000 (01:04 +0000)]
Make BranchMapper just map one name per call.
Move building a dict to the one callsite which actually wants that.

14 years agoRemove unused and unimplemented BranchMapper.bzr_to_git, and update docstring.
Max Bowsher [Wed, 4 Nov 2009 00:52:27 +0000 (00:52 +0000)]
Remove unused and unimplemented BranchMapper.bzr_to_git, and update docstring.

14 years agoAvoid spurious 'git-' being prefixed on branches whose names happen to end with ...
Max Bowsher [Wed, 4 Nov 2009 00:49:10 +0000 (00:49 +0000)]
Avoid spurious 'git-' being prefixed on branches whose names happen to end with 'trunk',
by tightening logic in BranchMapper._git_to_bzr_name.  Also document.

14 years agoSet a tag when touching a refs/tags/ ref with a commit command.
Max Bowsher [Tue, 3 Nov 2009 23:52:58 +0000 (23:52 +0000)]
Set a tag when touching a refs/tags/ ref with a commit command.

14 years agohttp read support for darcs-fast-export
Ian Clatworthy [Tue, 27 Oct 2009 05:49:55 +0000 (15:49 +1000)]
http read support for darcs-fast-export

14 years agoHandle multi-level branches
Ian Clatworthy [Mon, 26 Oct 2009 00:15:57 +0000 (10:15 +1000)]
Handle multi-level branches

14 years agoGet fastimport working on non-chk repositories again for bzr versions after 2.0.0
Ian Clatworthy [Sun, 25 Oct 2009 22:05:48 +0000 (08:05 +1000)]
Get fastimport working on non-chk repositories again for bzr versions after 2.0.0

14 years agodarcs-fast-export: refactor to a python class
Miklos Vajna [Thu, 22 Oct 2009 10:26:32 +0000 (12:26 +0200)]
darcs-fast-export: refactor to a python class

The code should be a lot more readable now and we can avoid function
names like open_() as well.

All testcases still pass.

14 years agodarcs-fast-export: add testcase for http export support
Miklos Vajna [Thu, 22 Oct 2009 10:26:11 +0000 (12:26 +0200)]
darcs-fast-export: add testcase for http export support

14 years agodarcs-fast-export: support exporting http repos directly
Miklos Vajna [Thu, 22 Oct 2009 10:25:39 +0000 (12:25 +0200)]
darcs-fast-export: support exporting http repos directly

14 years agohandle git ref names with slashes in them
Ian Clatworthy [Thu, 22 Oct 2009 07:55:22 +0000 (17:55 +1000)]
handle git ref names with slashes in them

14 years agoFix parsing error when a property is found after a multi-line one
Ian Clatworthy [Thu, 22 Oct 2009 07:36:00 +0000 (17:36 +1000)]
Fix parsing error when a property is found after a multi-line one

14 years agoChange the default inventory cache size to 1. For large projects, this reduces memory...
Ian Clatworthy [Wed, 21 Oct 2009 22:08:48 +0000 (08:08 +1000)]
Change the default inventory cache size to 1. For large projects, this reduces memory overhead and also speeds up conversion.

14 years agoupdate NEWS with recent changes
Ian Clatworthy [Tue, 13 Oct 2009 08:13:12 +0000 (18:13 +1000)]
update NEWS with recent changes

14 years agoAdd --user-map option to both fast-import and fast-import-filter
Ian Clatworthy [Tue, 13 Oct 2009 08:02:08 +0000 (18:02 +1000)]
Add --user-map option to both fast-import and fast-import-filter

14 years agoAdd a README explaining how to compile svn-fast-export.c
Ian Clatworthy [Tue, 13 Oct 2009 02:02:22 +0000 (12:02 +1000)]
Add a README explaining how to compile svn-fast-export.c

14 years agoignore generated executables
Ian Clatworthy [Tue, 13 Oct 2009 01:51:26 +0000 (11:51 +1000)]
ignore generated executables

14 years agoMerge Ted's improvements to svn-fast-export
Ian Clatworthy [Mon, 12 Oct 2009 07:52:44 +0000 (17:52 +1000)]
Merge Ted's improvements to svn-fast-export

14 years agoMerge fix for tags off the mainline
Ian Clatworthy [Mon, 12 Oct 2009 07:39:27 +0000 (17:39 +1000)]
Merge fix for tags off the mainline

14 years agobase file-ids on the basename, not path, as jam suggested. This improves the samba...
Ian Clatworthy [Mon, 12 Oct 2009 02:12:22 +0000 (12:12 +1000)]
base file-ids on the basename, not path, as jam suggested. This improves the samba import from 565M to 353M.

14 years agoAdding an address parameter.
Ted Gould [Sat, 10 Oct 2009 15:38:08 +0000 (11:38 -0400)]
Adding an address parameter.

14 years agoStore tags on entire branch ancestry, not just left-hand ancestry.
Max Bowsher [Fri, 9 Oct 2009 23:36:45 +0000 (00:36 +0100)]
Store tags on entire branch ancestry, not just left-hand ancestry.

14 years agofix date parsing bug found while importing samba
Ian Clatworthy [Fri, 9 Oct 2009 05:42:53 +0000 (15:42 +1000)]
fix date parsing bug found while importing samba

14 years agoAdding in a first rev parameter
Ted Gould [Fri, 2 Oct 2009 19:06:05 +0000 (14:06 -0500)]
Adding in a first rev parameter

14 years agoChanging authors to be from SF instead of localhost
Ted Gould [Fri, 2 Oct 2009 19:03:08 +0000 (14:03 -0500)]
Changing authors to be from SF instead of localhost

14 years agoSetting up makefile to use a little pkg-config
Ted Gould [Fri, 2 Oct 2009 19:01:37 +0000 (14:01 -0500)]
Setting up makefile to use a little pkg-config

14 years agomerge fix for empty patch name handling in darcs-fast-import
Ian Clatworthy [Sun, 27 Sep 2009 15:40:50 +0000 (01:40 +1000)]
merge fix for empty patch name handling in darcs-fast-import

14 years agomerge Harry's fix so directories aren't output in plain format
Ian Clatworthy [Sun, 27 Sep 2009 15:16:06 +0000 (01:16 +1000)]
merge Harry's fix so directories aren't output in plain format

14 years agodarcs-fast-export: no longer fail on empty patch name
Erik Hesselink [Wed, 23 Sep 2009 22:09:43 +0000 (00:09 +0200)]
darcs-fast-export: no longer fail on empty patch name

[ mv: testcase by me ]

14 years agoDon't emit directory info for renames operations when using plain format
Harry Hirsch [Fri, 11 Sep 2009 22:12:14 +0000 (00:12 +0200)]
Don't emit directory info for renames operations when using plain format

14 years agoMerged fix for directory handling in plain format
Harry Hirsch [Fri, 11 Sep 2009 22:12:03 +0000 (00:12 +0200)]
Merged fix for directory handling in plain format

14 years agoDon't emit directory info when plain format is specified.
John Whitley [Thu, 10 Sep 2009 05:29:28 +0000 (22:29 -0700)]
Don't emit directory info when plain format is specified.

14 years agomerge fixes for darcs-fast-import for better hg 1.3 interoperability
Ian Clatworthy [Wed, 9 Sep 2009 04:06:00 +0000 (14:06 +1000)]
merge fixes for darcs-fast-import for better hg 1.3 interoperability

14 years agodarcs: update d2x for Greg's hg-fastimport
Miklos Vajna [Wed, 9 Sep 2009 01:12:40 +0000 (03:12 +0200)]
darcs: update d2x for Greg's hg-fastimport

Previously I recommended my fork of
http://hg.opensource.lshift.net/hg-fastimport, but that's unmaintained
and Greg's hg-fastimport works out of the box with hg 1.3. Update the
code, documentation and testcases accordingly.

14 years agodarcs: handle timezones like +1300 in the importer
Miklos Vajna [Wed, 9 Sep 2009 00:31:07 +0000 (02:31 +0200)]
darcs: handle timezones like +1300 in the importer

darcs accepts +1200, but +1300 has to be turned to -1100. This is the
case when importing for example the asciidoc repo.

14 years agodarcs: make sure the working directory has no changes when verifying imports
Miklos Vajna [Wed, 9 Sep 2009 00:31:07 +0000 (02:31 +0200)]
darcs: make sure the working directory has no changes when verifying imports

14 years agomerge directory handling fix for darcs-fast-import
Ian Clatworthy [Tue, 8 Sep 2009 04:58:44 +0000 (14:58 +1000)]
merge directory handling fix for darcs-fast-import

14 years agodarcs: fix importing files in subdirs
Miklos Vajna [Mon, 7 Sep 2009 22:01:39 +0000 (00:01 +0200)]
darcs: fix importing files in subdirs

14 years agofix warning messages on non-utf8 names & emails
Ian Clatworthy [Sat, 29 Aug 2009 05:28:33 +0000 (15:28 +1000)]
fix warning messages on non-utf8 names & emails

14 years agoFix unicode email address parsing
Ian Clatworthy [Fri, 28 Aug 2009 14:28:10 +0000 (00:28 +1000)]
Fix unicode email address parsing

14 years agoHandle formatting of unicode emails
Ian Clatworthy [Fri, 28 Aug 2009 11:48:47 +0000 (21:48 +1000)]
Handle formatting of unicode emails

14 years ago--forward, not --reverse, in fast-export help
Ian Clatworthy [Fri, 28 Aug 2009 11:29:14 +0000 (21:29 +1000)]
--forward, not --reverse, in fast-export help

14 years agoimplicitly rename children on export when directory renamed
Ian Clatworthy [Fri, 28 Aug 2009 11:28:21 +0000 (21:28 +1000)]
implicitly rename children on export when directory renamed

14 years agoEnsure empty string property values stay as empty strings
Ian Clatworthy [Fri, 28 Aug 2009 11:20:24 +0000 (21:20 +1000)]
Ensure empty string property values stay as empty strings

14 years agoSanitize None revision properties to empty string
Ian Clatworthy [Fri, 28 Aug 2009 11:19:30 +0000 (21:19 +1000)]
Sanitize None revision properties to empty string

14 years agoMake sure renamed directories are found in file-id lookups
Ian Clatworthy [Fri, 28 Aug 2009 03:30:08 +0000 (13:30 +1000)]
Make sure renamed directories are found in file-id lookups

14 years agoHandle delete, rename then modify all in the one commit
Ian Clatworthy [Fri, 28 Aug 2009 03:06:20 +0000 (13:06 +1000)]
Handle delete, rename then modify all in the one commit

14 years agoMerge feature support including generation/parsing of commit-properties, multiple...
Ian Clatworthy [Fri, 28 Aug 2009 02:12:14 +0000 (12:12 +1000)]
Merge feature support including generation/parsing of commit-properties, multiple-authors and empty-directories

14 years agoTweak some diagnostic messages
Ian Clatworthy [Fri, 28 Aug 2009 01:16:04 +0000 (11:16 +1000)]
Tweak some diagnostic messages

14 years agoTeach fast-import-filter and fast-import-query about feature commands
Ian Clatworthy [Fri, 28 Aug 2009 01:00:11 +0000 (11:00 +1000)]
Teach fast-import-filter and fast-import-query about feature commands

14 years agotweak author formatting to use same smart rule as used for committer
Ian Clatworthy [Thu, 27 Aug 2009 23:09:14 +0000 (09:09 +1000)]
tweak author formatting to use same smart rule as used for committer

14 years agoadd revision count to 'Starting export ...' message
Ian Clatworthy [Thu, 27 Aug 2009 23:01:11 +0000 (09:01 +1000)]
add revision count to 'Starting export ...' message

14 years agoexport and import empty directories
Ian Clatworthy [Thu, 27 Aug 2009 22:57:45 +0000 (08:57 +1000)]
export and import empty directories

14 years agoFix feature checking
Ian Clatworthy [Thu, 27 Aug 2009 22:33:37 +0000 (08:33 +1000)]
Fix feature checking

14 years agoadd --plain option to fast-export
Ian Clatworthy [Thu, 27 Aug 2009 14:20:00 +0000 (00:20 +1000)]
add --plain option to fast-export

14 years agoValidate features are known before importing
Ian Clatworthy [Thu, 27 Aug 2009 13:26:40 +0000 (23:26 +1000)]
Validate features are known before importing

14 years agoMore intelligent empty directory removal (Tom Widmer)
Ian Clatworthy [Thu, 27 Aug 2009 13:06:50 +0000 (23:06 +1000)]
More intelligent empty directory removal (Tom Widmer)

14 years agoStore multiple authors and revision properties when defined
Ian Clatworthy [Thu, 27 Aug 2009 12:46:47 +0000 (22:46 +1000)]
Store multiple authors and revision properties when defined

14 years agoparsing of multiple authors and commit properties
Ian Clatworthy [Thu, 27 Aug 2009 11:47:04 +0000 (21:47 +1000)]
parsing of multiple authors and commit properties

14 years agoAdd missing tab characters to ensure that never born dirs are correctly removed durin...
Tom Widmer [Thu, 27 Aug 2009 10:23:38 +0000 (11:23 +0100)]
Add missing tab characters to ensure that never born dirs are correctly removed during each pass of parent directory pruning.

All tests (including new ones) now pass.

14 years agoAdd missing test cases for multilevel deletes in 2a dn 1.9-rich-root fast imports.
Tom Widmer [Thu, 27 Aug 2009 10:16:09 +0000 (11:16 +0100)]
Add missing test cases for multilevel deletes in 2a dn 1.9-rich-root fast imports.

14 years agoMerged latest changes from upstream.
Tom Widmer [Thu, 27 Aug 2009 10:14:24 +0000 (11:14 +0100)]
Merged latest changes from upstream.

14 years agofeature parsing
Ian Clatworthy [Thu, 27 Aug 2009 10:05:47 +0000 (20:05 +1000)]
feature parsing

14 years agomerge ghost handling fix from trunk
Ian Clatworthy [Thu, 27 Aug 2009 08:02:44 +0000 (18:02 +1000)]
merge ghost handling fix from trunk

14 years agofix broken tests
Ian Clatworthy [Thu, 27 Aug 2009 08:01:11 +0000 (18:01 +1000)]
fix broken tests

14 years agoFix ghost handling and improve progress tracking in fast-export
Ian Clatworthy [Thu, 27 Aug 2009 07:52:31 +0000 (17:52 +1000)]
Fix ghost handling and improve progress tracking in fast-export

14 years agoDefine feature names in one place
Ian Clatworthy [Thu, 27 Aug 2009 05:37:30 +0000 (15:37 +1000)]
Define feature names in one place

14 years agotweak proposed property formatting
Ian Clatworthy [Thu, 27 Aug 2009 05:25:24 +0000 (15:25 +1000)]
tweak proposed property formatting

14 years agoFirst cut at exporting additional metadata via 'features'
Ian Clatworthy [Thu, 27 Aug 2009 05:19:43 +0000 (15:19 +1000)]
First cut at exporting additional metadata via 'features'

14 years agoDefinition and formatting of multiple authors and revision properties
Ian Clatworthy [Thu, 27 Aug 2009 04:39:25 +0000 (14:39 +1000)]
Definition and formatting of multiple authors and revision properties