During the beta I have played around with deletion logging, gave feedback and had some questions.
Now shortly before the Domino V10 Launch event I think it is time to share some more detailed information.
As far I understood there are no changes planned between Beta 2 and GA.
Deletion Logging is implemented on lower database level and is activated per database. The settings are stored inside the database and do replicate.
Enable Deletion Logging per Database
Right now the only official way to enable deletion logging is to use the compact servertask with the -dl option (see examples below).
You can add up to 4 additional fields per database that you want to log. Those fields could be different depending on the type of database and the logging purpose.
The log does distinct between HARD deletes and SOFT deletes and also allows you to trace the deletion of design elements.
The compact operation looks like this (example):
local compact mail/nsh.nsf -dl on "$TITLE, Form, Subject, DeliveredDate"
After you enabled the deletion logging, each kind of delete is recorded in a central text file on the server.
IBM has chosen a text file for performance reasons. I would have wished they write this information into the deletion stub itself.
Here is how the standard log looks like and how it looks with the custom log fields.
In my example I am interested in the Subject, DeliveredDate. And also the Form of the document.
In addition for design elements I am interested in the design element name stored in $TITLE.
Those would be the type of fields I would add for example for a mail-database. The choice of $TITLE for design document delete can be quite helpful. The note class alone might not be sufficient to identify the design element deleted.
The resulting log files are stored in IBM_TECHNICAL_SUPPORT directory and look like this.
"20181006T130419,29+02","del.nsf","C125831B:004BC903","nserver","CN=Daniel Nashed/O=NashCom/CÞ","HARD","0008","45DF55C1:15FE5439C125831E:003CB1F4","$TITLE","TestFolder"
"20181006T130241,00+02","del.nsf","C125831B:004BC903","nserver","CN=Daniel Nashed/O=NashCom/CÞ","HARD","0001","92A7E32E:6A7A0FD9C125831B:0069F662","Form","Memo","Subject","Test Mail äöü / 电脑死机"
The name of the file has the following syntax:
delete__yyyy_mm_dd@hh_mm_ss.log
Like other files the currently active log file has a default name delete.log and the first line contains the name of the file for renaming it when the server is restarted (similar to console.log as shown above).
Here is a short description of the columns. The last columns depend on your configuration and the list is comma separated and has quotes around the values. Quotes are escaped accordingly.
- Timedate in the timezone of the server plus the timezone of the server at the end.
- Database Name
- Replica ID
- Process which deleted the note
- User who deleted the note
- HARD/SOFT Delete, RESTORE for a SoftDelete
- NoteClass of the document (for example 1 for a document, 8 for a view/folder)
- UNID of the document
After those standard fields you see the up to 4 custom fields that you have optionally specified with compact.
The first column always gives you the name of the field. The following column gives you the value.
The text elements and the total log line is limited. The current limits are 400 bytes per item and 4K for the whole log line.
This should be sufficient in most cases because we only need it to find the document.
The field types that can be used are Text, Text_List, RFC822_Text, or Time.
And those fields have to be present at the database when it is enabled!
The current log file encoding is in LMBCS (Lotus Multi Byte charset) which is the internal representation that Notes/Domino uses since day one to store text. But it would be difficult outside Notes do read this charset encoding.
There are plans for the next update to support other formats. But for now it is LMBCS.
Delete Log Application
All those files are located on the server and are not really admin friendly to read. So I am currently building a small application which imports the log files into a server database.
The same application also can collect the final log-files from each server and stores it a global database to archive those logs before deleting the physical file on disk.
So on the one side you will have a analysis database with all the logs in sortable and searchable way. And in addition you have full access to all the delete-log files in a central place.
The application has a scheduled agent which checks the current log file and also existing log files for importing and archives the older log files.
I have a first version implemented and I am still working on it. But I am planning to publish this database soon.
If someone wants to beta test the application, drop me a note. And I will also have it with me at the Domino V10 launch event next week in Frankfurt and also at the DNUG bleed yellow event the evening before.
Daniel