Zugg Software :: View topic

Posted: Thu Jun 24, 2004 6:34 pm

OK, this is embarassing and annoying all at the same time.

We went camping this past weekend. Then I got sick again. So, I haven't been paying much attention to things.

Well, the zuggsoft.com server decided to fill up it's disk last Friday just as I left for camping. So, for the past 6 days or so, the server has been screwed up, and more importantly, the Web-based order system has been screwed up. The order page would time out, but still charge the person's credit card. Some people got reg codes, some didn't. It's a complete mess and it looks like I'm going to end up refunding about $1000 worth of duplicate orders from people who kept resubmitting their order page with the timeout error.

The reason I'm posting is that I'm still sick and can't think straight. But what I need is help with a DOS BAT file to help purge my web logfile directory every day because that is the directory that is filling up and causing the disk to get full.

Basically, I have a directory called "logfiles" with files of the name format: EXyymmdd.LOG
where yy is the year, mm the month, and dd the day. For example: EX040624.LOG is todays log file.

So, the directory fills up with log files, one for each day. I need to run a BAT file every day to purge the old files so that only the most recent file remains.

It's been a while since I wrote BAT files and I don't remember any simple PURGE command. I don't have anything fancy like PERL either. It needs to be a simple BAT file. Can anyone suggest a script that will do this. It would really save me a huge headache trying to figure this out while I'm feeling bad.

And yeah, messing with 50+ screwed up orders this morning to figure out who has duplicate charges and stuff is really not what I wanted to spend this morning doing. On top of that, we have a power failure for an hour this morning.

I must still be cursed for some reason Sad

Thanks for any help you can provide.

GURU Joined: 28 Sep 2000 Posts: 156 Location: USA

Would this work?

GURU Joined: 28 Sep 2000 Posts: 156 Location: USA

You can replace the del at the end with dir for testing. =)

GURU Joined: 24 Sep 2000 Posts: 1237 Location: USA

@ECHO OFF
rem SET OLDERTHAN=%1
SET OLDERTHAN=1
IF NOT DEFINED OLDERTHAN (GOTO SYNTAX)

for /f "tokens=2" %%i in ('date /t') do set thedate=%%i

set mm=%thedate:~0,2%
set dd=%thedate:~3,2%
set yyyy=%thedate:~6,4%

set /A dd=%dd% - %OLDERTHAN%
set /A mm=%mm% + 0

:LoopDate
if /I %dd% GTR 0 goto DONE
set /A mm=%mm% - 1
if /I %mm% GTR 0 goto ADJUSTDAY
set /A mm=12
set /A yyyy=%yyyy% - 1

:ADJUSTDAY
if %mm%==1 goto SET31
if %mm%==2 goto LEAPCHK
if %mm%==3 goto SET31
if %mm%==4 goto SET30
if %mm%==5 goto SET31
if %mm%==6 goto SET30
if %mm%==7 goto SET31
if %mm%==8 goto SET31
if %mm%==9 goto SET30
if %mm%==10 goto SET31
if %mm%==11 goto SET30
if %mm%==12 goto SET31

goto ERROR

:SET31
set /A dd=31 + %dd%
goto LoopDate

:SET30
set /A dd=30 + %dd%
goto LoopDate

:LEAPCHK
set /A tt=%yyyy% %% 4
if not %tt%==0 goto SET28
set /A tt=%yyyy% %% 100
if not %tt%==0 goto SET29
set /A tt=%yyyy% %% 400
if %tt%==0 goto SET29

:SET28
set /A dd=28 + %dd%
goto LoopDate

:SET29
set /A dd=29 + %dd%
goto LoopDate

:DONE
if /i %dd% LSS 10 set dd=0%dd%
if /I %mm% LSS 10 set mm=0%mm%
for %%i in (*.*) do (
set FileName=%%i
call :PROCESSFILE %%~ti
)

set mm=
set yyyy=
set dd=
set thedate=
goto EXIT

:SYNTAX
ECHO.
ECHO USAGE:
ECHO DELOLD X
ECHO Where X is the number of days previous to Today.
ECHO.
ECHO EX: "DELOLD 5" Deletes files older than 5 days.
GOTO EXIT

:PROCESSFILE
set temp=%1
set fyyyy=20%temp:~6%
set fmm=%temp:~0,2%
set fdd=%temp:~3,2%
if /I %fyyyy% GTR 2069 set fyyyy=19%temp:~6%

:: +*************************************+
:: | This is where the files are deleted |
:: | Change the ECHO command to DEL to |
:: | delete. ECHO is used for test. |
:: +*************************************+
if /I %yyyy%%mm%%dd% GEQ %fyyyy%%fmm%%fdd% (
ECHO %FileName%
)

set temp=
set fyyyy=
set fmm=
set fdd=

:EXIT

Wizard Joined: 10 Oct 2000 Posts: 1551 Location: Australia

You're supposed to write log files to a different partition so that if they fill up the drive, you don't kill the server :)

Also, you could zip up the logs from your batch file instead of delete them, if you needed historical data.

I can't help with a dos batch file tho, my batch file experience is in linux hehe.

Posted: Fri Jun 25, 2004 6:06 am

Yikes Darker, that made my headache worse Wink

Seriously, thanks for the help, I'll give it a try.

Rainchild, yeah, ideally, but since I need to be able to get to the log files via HTTP, I like to keep them on the main server. But I only need the current day's log. This didn't use to happen before we turned on the extended logging. Before that there was only one log file. Not sure what changed to cause this daily file, but with Darker's script to purge the old ones, everything should be fine. I'll give it a try tomorrow.

Posted: Sat Jul 03, 2004 9:46 pm

Darker, I've been struggling with this. It doesn't work on the www.zuggsoft.com server. First there was a problem that the year is already returned as 4 digits (like 2004) so the 20%temp:~6% doesn't need the "20". I changed that and got it working on Windows XP. But then when I tried it on the WinNT server, it got to the line

call :PROCESSFILE %%~ti

and complained about the %%~ti flag not existing.

I also did a Google search and found 3 other sample BAT files...none of them worked even though they all said they were specifically for WinNT. And they were all *very* complicated...much more complicated than your script.

So, if you have a suggestion how to modify your script for WinNT, I'd really appreciate it. I can't believe that something like this is so hard.

If I can't figure this out soon, I'm going to end up having to write my *own* Delphi application to delete files older than a certain number of days. But this really seems like something that somebody must have already written somewhere.

After all, how else does anyone manage log files on their server? This seems like something that everyone running IIS would need in order to keep their log file directory cleaned up.

GURU Joined: 28 Sep 2000 Posts: 156 Location: USA

Does my version work on NT? It isn't as fancy and works differently but it seems like it would work.

Posted: Sat Jul 03, 2004 10:13 pm

Nevermind, I found a utility called XXCOPY that seems to do what I need. Here is a Google link that helped me with this problem:

http://groups.google.com/groups?q=windows+deleting+files+older+than+days&hl=en&lr=&ie=UTF-8&selm=4e377a72.0301201824.4137f289%40posting.google.com&rnum=2

or just find XXCOPY at http://www.xxcopy.com

GURU Joined: 24 Sep 2000 Posts: 1237 Location: USA

Sorry - wasn't my script... Google. Smile

Posted: Mon Jul 05, 2004 2:56 am

Yeah, *none* of the 4 scripts I got from Google (that claimed to work on NT) worked.

It seems that the BAT file syntax changed enough from one version of Windows to the next that it's now basically a worthless language to write useable sys admin tools in. I'd never even seen the %temp:~0,2% syntax, for example. It seems to be something added by Windows since I don't remember ever seeing it in regular DOS BAT files.

No wonder everyone is using Perl for their sys admin on Windows these days. Neither BAT files nor Windows Script files are consistent enough across OSes to be useful anymore. Bad Microsoft.

Anyway, I should have realized this and looked for an EXE file in the first place.

Oh, and Zor, I didn't try your method because it would easily delete the *current* file if the date changed between the file creation and the test. For example, you create a log file at 11:00pm and then the next day at 8:00am you run your script. Because the date doesn't match, the file would get deleted even though it's still the most current file.

Posted: Mon Jul 05, 2004 3:35 am

Turns out not even the XXCOPY was working last night!

Seems that people just post examples to newsgroups without ever even testing them first. The examples for XXCOPY didn't include the fact that it prompts by default when deleting ANYTHING, so when you run it in a BAT job, it fails. So I went ahead and added the command line switches to confirm the deletions. The stupid program actually tries to delete the entire DIRECTORY when it's done! Fortunately Windows doesn't let you delete a directory that isn't empty.

But why are people all excited over XXCOPY when it has bugs like this? It should never prompt for deleting a directory that it KNOWS is not empty.

Like I said before...I can't believe that this is SO HARD. Deleting old files is one of the most common tasks in system administration and nobody has solved this problem easily in the 15+ years we have had Windows/Dos? Unbelieveable.

Oh, and since this thread is already off topic, is it just me, or are newsgroups dead? newsgroups used to be my primary source of information on a lot of stuff but more and more often when I'm searching for questions the posts I find are very old, or else they are newer questions with no answers. Has everyone left the newsgroups? It's unfortunate because even though people probably like Web boards better, it was easier to index newsgroups in Google. Most of the web boards index very poorly (if at all) in Google.

Anyway, I've probably spent a total of 5 hours over the past couple days just trying to get the server to delete old log files. What a waste of time.

Posted: Tue Jul 06, 2004 9:38 am

TweakXP forums are usually pretty good. There's a lot of sections not related to their product, so it'd be worth a try.
http://forum.tweakxp.com/forum/

Magician Joined: 09 Oct 2003 Posts: 459 Location: USA

Because of the advent of spam address harvesters/posters, forums have become increasingly popular, because of the difficulty of scripts and bots to harvest and/or post into them. Also, forums have useability that newsgroups never thought possible when being designed.

Magician Joined: 09 Oct 2003 Posts: 459 Location: USA

Okay, the following is going to sound like a dumb question.... but!

Why don't you write a small delphi app that task scheduler can call that will clear that folder for you like you need?

Posted: Tue Jul 06, 2004 7:32 pm

That's not a dumb question at all Slicertool! Wink

In fact, I just checked the server this morning and the XXCOPY solution STILL ISNT WORKING!! I have no idea why this BAT file works fine interactively but refused to work when scheduled.

So, yes, actually I am going to have to waste a few more hours and just write my own command line application for deleting files older than a certain date. It's not going to be anything fancy, and it won't do any scheduling. It will just be a command line tool I can call from my current script file that processes my web logs each night.

I really should have done this in the first place. If I had any idea how much of a pain this was going to be then I would have just written my own app from scratch.

How many poor people have had to write this kind of application themselves. I'll certainly post it to Tucows when I'm done and try to contribute something useful back to the world. This kind of free tool should be readily available in my opinion.

Anyway, I'll post when it's done. But that's yet more time wasted on this stupid problem that should have been a 5 minute job.

Posted: Tue Jul 06, 2004 10:28 pm

Cool, that only took 1-1/2 hours to write, test, debug, and install. Now if this doesn't work I don't know what I'll do. But it's a nice little 53kb utility that works great in all of my testing, so now I just have to wait a day and see if it works in the scheduled BAT file on the NT server. Once it's confirmed to be working then I'll upload it to Tucows.

Here is my syntax help message for it:

GURU Joined: 28 Sep 2000 Posts: 434 Location: USA

Snicker.

That is what you get for using Winblows!

find /var/logs -name *.log -type f -mtime +2 -exec rm {} ;

Find files in /var/logs whose filename matches *.log that is older than 2 days, then execute rm on that file!

(That is, if I remember the syntax correctly)