Zugg Software :: View topic

Posted: Thu Mar 09, 2006 10:31 pm

OK, I'm sick of this. I've lost two days to CVS issues.

I use TortoiseCVS (and sometimes WinCVS) with cvs-nt running as the backend CVS server.

The problem is that way back when I created the CVS repository, I did a cvs add for a bunch of files that I shouldn't be tracking. Most of these are binary files created by Delphi. I don't want these files in the CVS repository.

The problem is that there seems to be no safe way to remove these files from the CVS repository without the files getting deleted when I do a "cvs update".

For example, I've got a file called CMUD.DSK. I don't want this in the repository, but I need this file in my development directory. If I use "cvs -f rm cmud.dsk" then it removes the file from the repository, but also removes the file from the disk directory...the file is now gone. Thank goodness for backups. I've complained about this before and never gotten a good answer.

Even worse, when I go to my Laptop and do a CVS update or a full clean checkout, it will also delete the CMUD.DSK file in the development directory again. I don't want CVS to keep deleting this file!!!

OK, so then I go to the master CVS repository directory on the server and remove the CMUD.DSK,v file. That stops the file from continuously being deleted from my source directory.

If I just remove the CMUD.DSK,v file from the repository in the first place, then the next time I do a CVS UPDATE on any of my computers, the file gets deleted from the local directory.

So, from what I can tell, this is the procedure to remove a file from CVS to stop tracking it:

1) Make a backup of the file
2) cvs rm -f file
3) cvs commit
4) On all other computers using CVS, do a "cvs update" to remove the file from each computer
5) Go to the server CVS repository and remove the file,v file
6) Go to each computer and restore the backup that was made in step 1

This is a PAIN! It should NOT be this hard to just tell CVS to stop tracking a file. I've got hundreds of binary files sitting in the CVS repository that I don't want to track. It's going to take me forever to get rid of them all.

I've considered just wiping out my CVS repository and starting from scratch, but then I'd lose all of the revision history stored for zApp and zMUD in the past.

Does anyone have any suggestions on a better way to do this? Like I said, I've wasted 2 days of programming time dealing with this and I'm getting sick of it.

I really like the CVS tools when they work, but when they don't work it's a real pain.

Wizard Joined: 14 Oct 2000 Posts: 1368

Posted: Fri Mar 10, 2006 1:36 am

The issue with directories and files you are having is exactly what made me turn away from CVS when I was looking for a version control program. Subversion is tons better and manipulating directories and file. I have been able to remove and set files to ignore. I do admit that CVS client tools are a lot more mature than Subversion's.

I think you can add those binary files to .cvsignore file(s). That should make CVS stop tracking your files. I don't remember what the command line for that is.

Posted: Fri Mar 10, 2006 2:08 am

Rorso: that's the problem...the "cvs rm" command will not work unless you specify the -f option. If the file exists and you don't use -f then it complains that the file hasn't been deleted yet. Basically, CVS is not set up to remove files from the repository without also removing them from your working directory.

OK, here is a solution. I got so fed up with this that I wrote my own program to do this.

My program is called CVSRemove (Download CVSRemove.zip). Here is the usage help that you get if you just enter CVSRemove without any arguments:

Posted: Fri Mar 10, 2006 3:42 am

Kewl! Glad you got that taken care of. There's also a TortoiseSVN for Subversion. My new job uses CVS, so I'm getting aquanted with it this week. I'm going to download that utility you wrote and put it on my main usb flash drive (I have several).

Thanks Zugg!

Posted: Fri Mar 10, 2006 4:17 am

Really? Maybe I'll have to check that out too. I've heard good things about Subversion but was so hooked on Tortoise that I didn't want to get rid of it.

Turns out I still have a lot of work to do on my laptop. A few weeks ago I had copied everything from my Development system to my laptop as a backup and so that I could program on my laptop.

Well, CVS and doing a network file copy DO NOT MIX! Every single CVS directory now has an incorrect Root file (the Development system uses :local: whereas the laptop needs to use :sspi:

Getting the laptop back "in synch" with the CVS server is taking a long time. I'm ending up having to checkout each repository again. And the other problem that I hate with CVS is when you try to checkout on top of a directory where the files already exist, you get error messages telling you to "move the conflicting file". There seems to be no way to *force* it to overwrite the existing files.

Yet more pain. Might be time for another utility program. Or maybe I'll just take this "opportunity" to check out SubVersion.

Posted: Fri Mar 10, 2006 4:24 am

Blah....problem is that I need a Windows Server. My linux box is no longer operating here, so my CVS server needs to run on Windows. For CVS I am using cvsnt. I don't see a Windows server for Subversion.

Man, all I want to do is share my source code on both my development system and my laptop. I don't want to share a network drive, because I need my laptop to work "standalone". I just want something where I can "submit" my changes to a "server" and have the server keep track of versions.

Am I using CVS in a totally wrong way here? Seems like I just don't get it. Everyone jumps up and down about how important version control is and how great and "mature" CVS is, but it just doesn't seem to work the way it "should".

How do "real" companies with dozens of programmers handle version control in Delphi? Maybe they are using a big expensive system like SourceSafe or something. But I thought that CVS was supposed to be the open source alternative.

I'm still pretty disgusted with all of this.

Wizard Joined: 10 Oct 2000 Posts: 1551 Location: Australia

In the past I've used windows grep to 'fix' all my copy/paste wrong directory grumblings or sever name changes (eg 'bob' our main server died and was retired and replaced by 'traci' so I had to grep for bob in all the cvs directories and replace with traci.

Wingrep is shareware from www.wingrep.com ... very useful program to have lying about - way better than the 'find in files' option of the MS sniffer dog.

Btw I'm fairly sure that tortoise moves the 'deleted' files to the recycle bin (don't take that as a guarantee tho, backup first)... so you should be able to do that cvs remove/commit, then just restore the deleted files from the recycle bin and it should stop tracking them - I think... I've never had reason to actually test that theory tho :)

Posted: Fri Mar 10, 2006 4:50 am

There are two ways of running Subversion. The most common one is using Apache. The other way is by making svnserve.exe run as a Windows NT Service. This is how I run all of my Subversion servers on Windows. You don't need a server version of Windows to run it either.

I'm creating a management system for Subversion. I did this because I got tired of trying to remember the command line switches. This thing is not done yet, but you can see what I've done at https://sourceforge.net/projects/svnmanagerlib.

On Delphi and Source Control, I know a lot of companies use JEDIVCS ( http://jedivcs.sourceforge.net/ ) It has both a stand alone client and integration with the Delphi IDE. I used this back when I was still programming in Delphi. The other one is Team Coherence ( http://www.qsc.co.uk/VersionControl.htm ).

Email me if you want to know more about running a Subversion server on Windows. I have a lot of experience doing that.

Posted: Fri Mar 10, 2006 5:03 am

Man, what a night. After getting half-way done recreating all of my source directories on the laptop, I found a macro in WinCSV for changing the CVS root. I could have saved a lot of time if I had seen that earlier.

Also, it seems that there *is* a subversion server for Windows. I'm not sure I like having stuff stored in a database though. I think I'll stick with CVS for now. At least as I've seen tonight I can always easily manipulate the respository since it's all just normal text files.

Posted: Fri Mar 10, 2006 5:09 am

Rainchild: Actually, sometimes CVS bybasses the Recycle Bin, which can be a real shock! It happens when you remove a file from the repository (the ,v file). Then when you do an Update, CVS deletes the files in the source directory and doesn't put it into the Recycle bin. Bad bad bad.

And yes, I have a grep program, but I've never gotten comfortable enough with it to actually modify files. I just use it to find stuff and not to change stuff.

bortaS: JediVCS (the new open source version of the former FreeVCS) also uses a database backend. For some reason I'm just nervous about that. I also discovered that the "hooks" in Delphi for version control systems (SSC API) is only available in the Enterprise edition of Delphi, which I don't have (it's way too expensive). Maybe when the new company takes over Delphi they'll realize that us smaller developers want version control too.

Anyway, back to Subversion...The only database server that I have running is MySQL. Does the Windows version of Subversion support MySQL? Also, if I used Subversion I'd like to convert my existing CVS repository to maintain the version history. I saw a Python utility for that. I have Python installed, but I've read some issues using cvs2svn on Windows. Do you have any experience with that?

Posted: Fri Mar 10, 2006 5:10 am

Do you have an URL for the subversion server that you found? I only know the one that comes with the subversion official download. Subversion has two types of repositories, Berekley Db and FSFS ( File System ). I always use FSFS, which works just like CVS.

Posted: Fri Mar 10, 2006 5:21 am

I think it's the official site: http://svn1clicksetup.tigris.org/

Also, now that I've found the macro for renaming the CVSRoot for a project, I've converted all of my :local: links on my main development system to use the same :sspi: that the laptop uses. It works just fine for the local host and now the CVS files are the same on both systems so if I accidentally copy directories again in the future, then I shouldn't screw up the CVS stuff.

Anyway, let me know if converting the CVS repository to SubVersion is easy. At this point I'm really tired of working on all of this. Seems like I should have had this all working already from back when I did CVS for zApp/eMobius. Never expected to waste two days on getting this all working again.

Posted: Fri Mar 10, 2006 5:47 am

Well, if CVS is working for you, I don't recommend moving to Subversion yet. I would rather you have you working on cMUD instead. Twisted Evil

If you do feel the urge to change to Subversion, there is a python script that does that for you. It's called cvs2svn http://cvs2svn.tigris.org/

I've never heard of the svn1clicksetup thing. I've always done it by hand from the zip file download. Ahh, the things one discover every day. Wink

Posted: Fri Mar 10, 2006 5:51 am

Just took a quick look at SubVersion. I'm not sure it's going to be any better than CVS. So maybe you can answer these specific questions:

1) If I accidentally Add a file that I later decide shouldn't be version controlled (like an EXE file, for example), how do I remove that file from SubVersion. Seems like it works the same as CVS and always deletes the local copy of the file. I want a way to only remove the file from the SubVersion server and not touch the local copy.

2) I didn't see the method for specifying file extensions to be ignored. You mentioned that it was easy, but I didn't see it anywhere. Also, in CVS I've learned that the .cvsignore file is case-sensitive. In other words, if I tell it to ignore .exe files, it will still add .EXE files. Sometimes Delphi or other tools change the extension to uppercase and this has caused a lot of CVS headaches. I'd like there to be a setting for case-insensitive systems like Windows where ignoring .exe would ignore any case of .exe or .EXE etc.

3) Still trying to learn how to convert a CVS repository to SVN on Windows. Is there a site dedicated to helping Windows users with this? Honestly, I don't even have a TAR program for Windows laying around, and the Python script is only available as a .GZ.TAR file. Not something that WinZip will handle.

Think I'm going to bed now though. I'm totally brain fried.

Posted: Fri Mar 10, 2006 8:11 am

Well, I couldn't sleep. Kept thinking about this stuff. So I went ahead and installed SubVersion to look at it first hand.

First of all, it *was* really easy to install and set up. The link I gave above to the http://svn1clicksetup.tigris.org/ file really was easy. Once I got the Windows Service set up, I imported a test project and started messing around. The news was both bad and good.

The bad:

1) SubVersion shares many of the problems that I've had with CVS. For example, although there is a really easy way to browse the repository and remove stuff, when you then apply an Update to your working directory, it will always remove the file just like with CVS. And, just like CVS, removing files in this way does NOT put them into the Recycle Bin. Bad Bad Bad!!!

2) The Global Ignore list is a simple edit box with patterns separate by spaces instead of a list box. It makes managing a large list of ignored file extensions more of a pain then with CVS. Also, file extensions are still case sensitive, so no difference there.

3) Still haven't gotten my CVS stuff converted to SubVersion...that's a job for a day when I'm more awake.

The good:

1) The feature that will probably cause me to convert from CVS to SVN is the integrated diff/patch support. I've got a set of common changes that I have to make to the Developer Express components every month when I download the latest update. It's a real pain to do this manually. With Subversion, I should be able to create a patch file and/or merge my changes with there's. This could save me a lot of time and make it a lot less painless to update the DevExpress components each month. I'm a bit nervous about an automatted patching and merge system like this, but it's worth playing around with. I've looked for other Diff/Patch tools and while there are many Diff programs for Windows, there isn't much in the way of Patch utilities.

2) The other nice feature is that the Add is now recursive. This makes my automatted nightly scripts easier since I can just automatically do an Add for my main project directory, and then a Commit. With CVS you couldn't add an entire directory, which was a real pain.

3) So far the speed of SVN seems better than CVS.

So, it's a close call. I'll play with it more tomorrow. If I find a way to easily convert my existing CVS repository then I'll probably make the switch. Since I got my CVS repository cleaned up today with the utility program I wrote, I already have a good idea what files need to be ignored.

Posted: Fri Mar 10, 2006 8:10 pm

I've got the cvs2svn conversion running now. I was running an old (v6) version of WinZip. I found a copy of WinZip v8 that my registration code still works for and it was able to handle the tar.gz file.

Interesting side note: WinZip was one of the few other Shareware companies that started at about the same time as Zugg Software and also had a free upgrade policy. I noticed that with WinZip 10.0 they have also stopped the free upgrades. Thought that was interesting.

Anyway, once I figure out how to run the cvs2svn script with Python it then died at the "sort" step because I hadn't fully read the instructions and installed the GNU sort program. So I downloaded the UnxUtils package and got a copy of sort.

Now the conversion is on the final step of running the various SVN commit commands. It's on commit 12 of 135 so this is going to take a while.

Doing the "checkout" is going to be a pain. Like CVS, it will not overwrite existing files that are not version controlled. This seems like a stupid restriction. After all, it has the tools and knowledge needed to checksum the existing files and see that they are exactly the same as the current version in the repository. And if the files are identical then checkout shouldn't complain about overwriting them.

It still boggles my mind at how hard this all is to set up for existing projects. Sure, it works great for *new* projects, but putting existing projects into either CVS or Subversion is just a big pain.

So, I'll have to do a "checkout" to a fresh directory, then copy the other binary files that are not version controlled on top of the new directory and then rename directories. I'll probably leave the old CVS stuff around until I know this is all going to work.

If I'm more careful this time with how I do the checkout and be sure to specify the full SVN server URL (and *not* use localhost), then I can hopefully just copy the resulting directories over to the laptop when I'm done.

While I'm waiting for the importer to finish I'm going to go read some more articles on Subversion and see if perhaps I should have my project directories set up in a different way that would make this all easier. Delphi doesn't make this easy since it wants everything in a single directory, but maybe I'll learn a better way to organize this.

Now on commit 16 of 135. Wow, this really is going to take a long time :(

Posted: Fri Mar 10, 2006 8:48 pm

Grrr.....step 23 of 135 gave an error about some file that it couldn't encode. Another 30 minutes down the tubes. Starting over *again*

Wizard Joined: 14 Oct 2000 Posts: 1368

There is a Tortoise for subversion as well: http://tortoisesvn.tigris.org/

Posted: Fri Mar 10, 2006 10:13 pm

Rorso: Right, bortaS already mentioned it...that's what I'm using.

The cvs2svn importer failed again. I'm going to have to do this one module at a time. I just can't sit here and have it die on step 30 of 135 after an hour. Who knows how many other problems there are. There is an option to start the conversion at a specific step, but this last step 8 seems to be where all of the work is (when it needs to do the 135 commits). There is no way to start in the middle of the commit list that I can find.

So now I've started the conversion over (again) and am doing it on a module by module basis. At least this way I shouldn't have to keep starting over.

The people who said converting CVS to SubVersion and keeping your revision history was a pain were RIGHT!

Posted: Sat Mar 11, 2006 3:49 am

Well, I'm just about done with the CVS -> SVN conversion. The module-by-module conversion was definitely a better way to go. Basically it involves exporting a CVS module into a SVN "dumpfile" which you can then load into SVN. I'll probably keep these dump files around for backup for a while.

I'm now in the middle of recreating all of my "working directories" by making a clean checkout from SVN and then copying over the non-versioned files from the previous directory.

I have to plug an incredibly useful tool that I've been using: Directory Opus from GP Software (www.gpsoft.com.au). This is a Windows File Explorer replacement that I've been using for a while. It's so much better than the normal Windows Explorer I can't possibly give it justice here. But one of the features is the ability to view a directory in "flat" mode, which makes deleting all of the old CVS subdirectories a breeze. I don't know what I'd do without this tool. It's been extremely stable and comes *highly* recommended.

Once the working directories have been created, I'll then do some build tests to make sure I didn't lose anything (I still have Acronis Images from each of the past several days just in case). Once it looks like it's working, then I'll copy all of the new directories to the laptop and see if that finally takes care of it.

Whew...this has sure taken a long time. That's about 4 days lost out of my programming schedule. But hopefully it will end up saving more time in the long run.

Thanks to everyone for all of your suggestions. It's a big help for me when you make these kind of suggestions about stuff that you are more familiar with than I am. The more I use Subversion, the more I'm liking it better than CVS. Still has quirks, but maybe I've just dug around enough in the guts of the system now to be more comfortable with it.

Posted: Sat Mar 11, 2006 4:27 am

OK, what the hell is wrong with SubVersion??!!

I thought it handled binary files. Well, all of my binary DFM Delphi form files are corrupted! What kind of version control system corrupts files like this? I'm not sure how I'm going to fix this. I think the problem is that newer DFM files are text and older DFM files are binary so I think SubVersion is trying to treat them as text files.

But I thought that one of the big selling points of SubVersion is that it treats everything as binary and does binary diffs? Whatever it did, it sure screwed up things bad. Restoring all of my DFM files in all of my various subdirectories is going to be another royal pain.

As usual, I appear to be cursed.

Posted: Sat Mar 11, 2006 5:34 am

Looks like SubVersion is fine. It was the cvs2svn conversion script that messed up the DFM files. Any other binary files converted by this program are problem also messed up. I'll have to check.

Fortunately, I had good backups of the previous working directories. Using the life-saving Flat-view feature in Opus, I was able to copy over all DFM files from my backup to my new directories. I then wrote my own Delphi program to recursively search all of the directories and convert any binary DFM form files into the newer text format. Then I just committed these changed files to their new SVN repositories.

I'm still checking for other corrupted files, but it's looking better now.

Posted: Sat Mar 11, 2006 6:14 am

When you run into problems, you never seem to get the easy ones. At least you know how to create little utilites to help you get what you want done.

"1) SubVersion shares many of the problems that I've had with CVS. For example, although there is a really easy way to browse the repository and remove stuff, when you then apply an Update to your working directory, it will always remove the file just like with CVS. And, just like CVS, removing files in this way does NOT put them into the Recycle Bin. Bad Bad Bad!!! "

Yup, I thought there was a way around this, but I was wrong. I don't know why the Subversion team keeps overlooking this. It is a very aggravating thing. Maybe I should go pester them on the mailing lists or something.

Posted: Sat Mar 11, 2006 6:35 am

Also corrupted RES and DRC binary resource files. It looks like that might be it. But I'll save these backups just in case.

But my projects seem to compile and run now...finally.