Archiving Projects and Removing Locks

I'm having a hard time removing locks from users in order to do an archive of the Projects in the database. The program I have executes each night at 2am when no one is here, but if a user forgets to log out, it needs to remove any locks they have on the Projects. I have a function in there that removes the necessary locks. The problem is (and it is not consistently happening each time or anything), sometimes the function that actually archives the Projects executes, and I get an error telling me it couldn't lock a Project because it is locked by someone. Here is the exact error:
 

I- DXL: Item /ProjectName locked (shared) by 1 user
    UserName
Failed to lock archive item (ProjectName) for inclusion in archive

The thing is the function that removes the locks seems to be working. the thing logs the locks it attempts to remove and those that it does remove, and they are always the same. The user and project item are in the list of locks it says it removed. Here is the function that removes the locks:

bool clearAllLocks()
{ 
    LockList lList    = null;
    Lock     lockItem = null;
    string   errMess  = null;
    string   thisUser = null;
    Skip     lockSkip = null; 
    bool     lRemoved = false;
    int      count    = 0 
    
    logMessage("Clearing database locks...\n");
        
    lList = getLocksInDatabase(true);
        
        if(null == lList)
            return true;
            
    for lockItem in lList do
    {
        if(null == lockSkip)
            lockSkip = create();
            
        put(lockSkip, count, lockItem);
        count++;
    }
    
    if(null == lockSkip)
        return true;
    
    for lockItem in lockSkip do
    {
        thisUser = lockItem.user;
        
        logMessage("Attempting to remove lock for " thisUser "\n");
        errMess = remove(lockItem);
            
            lRemoved = lockItem.removed;
            if(!lRemoved)
            {
                logMessage("Could not remove lock for " thisUser "\n");
                return false;
            }
        if(null != errMess)
        {
            logMessage("Error Removing Lock: " errMess "\n");
            delete(lList);
            delete(lockSkip);
            return false;
        }
        else
            logMessage("Succesfully removed lock for " thisUser "\n");
    } 
    delete(lList);
    delete(lockSkip);
    return true;
}

One more thing. The user that executes the program is a "Database Manager" user. This means the user has the rights to retrieve a list of locks from the entire Database from all users and remove them.

Anyone have any ideas what could be going wrong here?

Re: Archiving Projects and Removing Locks
woodpryan - Fri May 01 12:51:04 EDT 2015

I was able to answer my own question, actually. Seems to me that it is a bug in DOORS. Here's the scenario:

1. When the program is executed, it removes a lock from userA and does the archive

2. UserA comes in the next day and never closes DOORS, but continues working in the database, particularly in the same Module from which the lock was removed the day before

3. When the program runs again that night and UserA is still in the Database, the locks they have on the data are not retrieved in the LockList.So, when it goes to archive the Project, we get an error.

Five tests were executed to verify this theory. Very interesting stuff.

Re: Archiving Projects and Removing Locks
Mike.Scharnow - Sat May 02 04:53:21 EDT 2015

I would take a different approach: announce that the database will be shut down every night and do the backup after the database is restarted again.

This way you will get rid of old locks and of used licenses

Re: Archiving Projects and Removing Locks
woodpryan - Mon May 04 10:21:03 EDT 2015

Agreed. The official policy is to exit DOORS at the end of the day before leaving. Unfortunately, the policy isn't always followed and is not enforced strongly enough. The customer doesn't want their backup to be put in jeopardy because someone forgot or didn't care to log off before going home. So, here we are.

Re: Archiving Projects and Removing Locks
Mike.Scharnow - Mon May 04 10:41:29 EDT 2015

Well, you want to remove locks. So the developer who did not close their module will lose their connection anyway through your backup script.

So, the easiest way to remove all locks is to restart the database, and this has the same effect as your script plus you will have a consistent backup if you do it after the restart. So why bother with the unlocking mechanism?

Re: Archiving Projects and Removing Locks
Mike.Scharnow - Mon May 04 10:48:18 EDT 2015

Edit: I see that my statement "So the developer who did not close their module will lose their connection anyway through your backup script" is not valid in the DOORS version that you use, according to your statements 2 and 3; it seems that the user is not informed about the nightly removal of the lock. But these statements also show that the internal state of the module is severely messed up in your scenario. IMHO this is way worse than a clean DB shutdown.

Re: Archiving Projects and Removing Locks
woodpryan - Mon May 04 11:10:51 EDT 2015

The user is aware of the nightly backup and lock removal, and should be logging out every night. I have informed the customer that management needs to start getting involved and cracking down on the users that don't follow policy. The script, of course, logs the user information for those from which it must remove locks. Hopefully, they will heed my warnings about this.

So, are you saying that the script removing the locks could possibly cause corruption in the Database? Also, off the top of your head, are you aware of a way in DXL to restart the Database? I didn't know we could do that.

Re: Archiving Projects and Removing Locks
Mike.Scharnow - Mon May 04 16:03:27 EDT 2015

Well, at least the behavior that you describe seems to hint that DOORS does not behave as expected. A worst case would be that a UserB is able to open the module in exclusive edit mode at the same time as UserA.

No, you would not shut down the database in DXL, that would break the execution in the middle of the script. You would have a windows batch file (or cmd or PowerShell) that stops and restarts the database server and then does the file system backup and then it would start a doors client which does the project archive.

Re: Archiving Projects and Removing Locks
woodpryan - Mon May 04 16:54:27 EDT 2015

What would be the ramifications of shutting down the Database while someone is logged in? I wonder what would happen from their perspective. Would they just get that message about the connection having been reset? After I click 'OK' on that, if I remember correctly, my DOORS client is restarted. Do you know, off the top of your head, what a batch command to restart the database might look like? If not, I can try and figure it out. Thanks for your help thus far.

Re: Archiving Projects and Removing Locks
Mike.Scharnow - Tue May 05 03:35:58 EDT 2015

I just tested this on my pc:

- open a module exclusive, do some unsaved changes

- restart the doors server

- the user gets the message and after clicking "ok", the application is terminated.

The batch file consists of a net stop and a net start, see http://www-01.ibm.com/support/docview.wss?uid=swg21597508. You have to find the real internal name of the service. try a "net start" on the console of the DOORS server, it will give you the names of the started services.

Re: Archiving Projects and Removing Locks
sekrbo - Tue May 05 09:46:39 EDT 2015

My standard backup script:

date /T > D:\DOORS_backup\DOORS_backup.log
time /T >> D:\DOORS_backup\DOORS_backup.log
net stop "DOORS DB Server 9.5" >> D:\DOORS_backup\DOORS_backup.log
rd D:\DOORS_backup\data /s /q >> D:\DOORS_backup\DOORS_backup.log
xcopy "D:\...\data" D:\DOORS_backup\data /s /e /i >> D:\DOORS_backup\DOORS_backup.log
net start "DOORS DB Server 9.5" >> D:\DOORS_backup\DOORS_backup.log

Note 1: the specific name of your DOORS server service may differ and you have to find where your DOORS data is stored.

Note 2: Do NOT use this if you're using DWA as well, that requires shutting down the DWA components and restarting them as well!

/sekrbo

Re: Archiving Projects and Removing Locks
Chris Annal - Fri Jun 26 10:11:35 EDT 2015

Well, I took a different approach with the script I occasionally run to perform the same task. It can be run as a batch, but it simply skips over any locked project and logs the username of the offending party. At the end of the archiving process, it sends an email list to me of all the projects that were archived, as well as any that were locked at the time. An email reminder to that user is usually sufficient enough. They shouldn't be leaving DOORS open and running when they leave at night, anyway.

Here's the script, if you think it might be useful:

//Archive_Projects_CheckLocks.dxl
/*
Archive projects, after first checking to make sure they aren't locked - email results
to defined recipient from email account set up for this purpose
*/

//Created 09/07/2011 -  C. Annal
/*******************************************************************************
 * Global constants
 */
              
const     string    targetAddress = "yourname@yourcompany.com" // replace yourname and yourcompany with actual address
const     string     emailServer = "mailserver.yourcompany.com" // replace mailserver and yourcompany with actual email server your company uses
const     string     emailAccount = "yourname@yourcompany.com"  // Your company may set up a special email account, for instance "DOORS_DBA@yourcompany.com"
const    string     fromDescription = "DOORS"
const    string     subject = "Archive Results"

/*******************************************************************************
 * Global Variables
 */
Lock lockItem
LockList lcklist
string sProjectName
string message     = ""
Project pX
Buffer bx1=create, bx2=create

/*******************************************************************************
 * Function Declarations
 */
bool bLockedByAnyUser=true, bLocked

/*******************************************************************************
 * Script
 */

for pX in database do {  
sProjectName = name pX   
Project parent = getParentProject(pX)
if(null parent && !isDeleted pX){
   lcklist = getLocksInFolder(pX,true,bLockedByAnyUser)
   bLocked=false
for lockItem in lcklist do {
   bLocked=true;
   string lckuser = lockItem.user
   bx1+=sProjectName " locked by " lckuser "\n";
   break
}// End for lockItem in lcklist do
   if(bLocked)continue
   bx2+= sProjectName " was processed \n"
// YOU MUST CREATE A FOLDER ON YOUR HARD DRIVE TO SEND THE ARCHIVES TO
// THIS IS CURRENTLY "c:\\DOORS_Projects" in this script. Change it as needed or create a folder named "DOORS_Projects" on your "C" drive.
   string proj_archive = archive (sProjectName,"c:\\DOORS_Projects\\"(sProjectName)".dpa",false,true,allBaselines,false)    
if (!null proj_archive)
    {
        ack proj_archive
        halt
    }// End !null proj_archive
}// End if(null parent && !isDeleted pX)
}// End for pX in database do
message = bx1 "\n" bx2""
sendEMailNotification(
    fromDescription,
    targetAddress,
    subject,
    message)
//halt

Re: Archiving Projects and Removing Locks
woodpryan - Fri Jun 26 13:03:11 EDT 2015

I agree with you, Chris. My customer, however, did not share our opinions, and I could not convince them otherwise. My requirements were very clear, and I had to come up with a solution. I did come up with one, and they have been using it for a while now.