Migration from IBM Domino to IBM Verse

1216 - Smooth Migration from IBM
Domino to IBM Verse : The Vital
Metrics You Do Need
Jérôme Deniau, move4ideas / September, 16th 2016

About Jérôme Deniau
• Co-founder & CTO at
• Member of Penumbra – an international consortium of selected Business Partners
pooling their talents and resources
• Designer of for Domino in 2006
• IBM Design Partner member
• Certified IBM/Lotus Instructor since 1993
• Certified Advanced Domino Administrator and Developer since Lotus-Notes R3
• Jerome.deniau@move4ideas.com

Abstract
• IBM Domino server is having an exceptional functionality and rich
features which fit perfectly for customer and their very own business
needs.
• IBM Verse defines a #NewWayToWork that comes with Social
powerful features you can use and activate without compromising your
IBM Domino investment.
• How to migrate your IBM Domino server mail system to IBM Verse
simply without no hassle downtime and make your end-users happy
with the migration process.

Agenda
• Why Are We Here
• Pre-requisites
• W4
• Metrics

Why Are We Here
• Audience
 Main target: IT Department and C-level but it is not limited
• Architecture change
• Migration headaches
• Where is my starting point
• #NewWayToWork
• Keep It Simple (but NOT) Stupid
• Keep in mind the 4 V’s (Value/Variety/Velocity/Volume)

Prerequisites
• Before Rush
 Knowledge
 Hard work
 Attitude

Prerequisites - Knowledge
• This is the very first step
 Can’t do without!
• Source
Have a good knowledge of your Domino environment and
architecture
• Target
 Know your target and Procedures
 On-premise
 Hybrid
 Pure-cloud

Prerequisites – Hard Work
• Prepare your team and your system
 Complete checklist of your system
 No rush
 The simpler the better

Prerequisites - Attitude
• Last but not least
 Be positive
 Be invested
 Be inventive
 Be an evangelist
• Make it a success

W4
• The W power
 The “Why We Should Move” is really far behind us by now
• The four vital points:
 Who
 What
 Where
 When
W4

W4 – What For?
• You will be able to define a set of population to migrate them altogether
When
WhatWho
Where

W4 – The Who Dilemma
• Active vs inactive users
 What defines active people vs inactive people
 One can be active without answering mails
 Database Activity
 Mail rules
• Relations: people who collaborate, work together
• Domino applications and mail agents
• External users
Who

W4 – The What Dilemma
• Smartphones vs IBM Notes addicts
• Attachments: Enterprise data
• Huge databases
• Sensitive data
What

W4 – The Where Dilemma
• Limited with IBM Verse for the cloud version
 Asia Pacific
 Europe
 United States
• For this session metrics will take in account these targets instead:
 Hybrid
 On-premise
 Pure cloud (private or public) Where

W4 – The When Dilemma
• Last step at last
 Once populations are defined
 Deployment plan to run the migration at the right time for the population to
migrate
• Off-hours
• Off-peak
• Avoid Monday morning burst When

W4 – The Big Picture
When
WhatWho
Where

Use Case
• We want to migrate only active users!
 How to get this information from Domino ?

Migration from IBM Domino to IBM Verse

Active vs Inactive Users
• You do need to distinguish between false active and false inactive
 Comparing databases sizes
 Domino built-in tools for accurate information
 Database properties: user activity detail
 Domino log.nsf ‘Database Usage’ view
 Domino activity logging (activity.nsf)
 License Tracking (userlicenses.nsf)

• Quick and dirty metrics based on database files sizes
 Export the mail files view from IBM Administrator on regular basis
 Too simple and quite confusing
 CTRL-C + Paste into a spreadsheet or better use IBM Watson Analytics

• Less dirty: log.nsf using the Usage views
 Statlog should run regularly

• Quick and dirty (cont.)
 Database sizes metrics are insufficients
 View indexes sizes are missing (you can get rid of them with
purge or compact –D)
 Take care of:
o Updall –c (in clustered or non clustered environment)
 Need a tool to analyze correctly the information

• Database user detail
 No_Force_Activity_Logging=0 and Statlog task should run
 Pros:
• simple to use
• C api calls can be used to grab the information
 Cons:
• Results are on a database basis
• Erratic problems
• Limited use due to the 32kb limit (Wait for the RNext 16MB)
 Solution: use scanEZ from Ytria

• Database user detail samples
• IBM Notes
• Ytria DatabaseEZ

• Domino log.nsf
 Information is set by the Statlog task
 Add Log_Sessions=1 or 2 in notes.ini
 Remove Log_Disable_Session_Info=1 if exists in notes.ini

• Domino activity logging
 Simple configuration
 Domino.Notes.Session
 Domino.Notes.Database
 Validation: show server
 IBM Technote for retention period
 #1230016 (from 2010) default is 7 days so:
o Log=logfiles/log.nsf,1,0,7,40000,[1-6]

• Domino activity logging (continued)
 Run the Activity analysis tool

• License Tracking
• Simple configuration
• Adminp is required to aggregate (so take care of that)
• All views under ‘Active Users’ can be a good starting point

• Conclusions
 A user who does not send mail can be active
 Message Tracking Collector?
 Automatic or manual reports in reports.nsf:
o Top 25 xxx
o Messages size
o Message volume

Tips
• Mail Tracking Collector retention period
 Default is 30 days
 Accommodate this value according to your needs

Use Case
• We want to migrate people involved in ‘communities’.

Relationship
• How to define groups of people working together
 Do we have metrics out of the box?
 Technically:
 Complicated if we talk about Communities detection for Domino
 Complicated if talking about Dynamic Graphs
 Complicated to implement Louvain method
• So what is available and simple to use?

Relationship
• Message Tracking Collector
 Yes but what for? Better compare manually the information
 Inefficient for relationship but fields can be useful

Relationship
• Increasing Domino logging for mail:
 Log_MailRouting=30
 Result:
 Once again limited: but
 Data is stored in log.nsf in the “Mail Routing” view
o If LOG_MailRoutingToMiscEvent=1 is not specified
 Data is stored in text file(s) console log(s) in <Domino data
Folder>/IBM_TECHNICAL_SUPPORT

Relationship
• Domino Mail Journaling
 High volume on some sites
 Needs replication between servers
 Caution due to high volumes and data retention delays
 Needs development to build a map for user relationship
 Do not encrypt $TUA nor $MessageID
 $TUA field can help you group mail exchanges on one subject amongst a group of user
o Beware: you can have multiple $TUA fields

Relationship
• Activity Logging
 Domino.Mail entry type
 As-is needs a bit of design to make it work the way you need it
 Use the activity logging analysis tool from IBM Administrator
 Deposited view
 Processed view

Relationship
• What the analysis tool does is:
 Get the documents from log.nsf
 Form=“Mail Routing”

Relationship
• Organizational Units can help
 Group people according to their OU’s
• Domino mail server
 Group people according to their home server

Relationship
• Conclusions
Not really an out of the box functionality
 Need a specific development
 Or a third party tool

Use Case
• We want to distinguish between ‘Human’ and ‘non Human’
users

Domino Applications & Mail Agents

• In a wonderful world
 Designers are using a ‘fake’ or non user mail account when the application or
the agent sends a memo
 Designers can use two users information
 Return-path: in case of error (Field Principal or From)
 User reply address: for a normal reply (field ReplyTo)
 (optionally) define a fake mail address in agent security property field “Run on behalf
of”
 Designers have used NotesLog or Log classes and an agentLog database

• Activity logging
 Insufficient, lacks of information on agent type and
 Does not generate information for the following agents types:
o Web agent
o Manually run agent from the IBM Notes client
o Scheduled agents that run locally on a client

• ICAA application supported
• Keep the agents and applications on premise or Hybrid Verse with
SoftLayer
• Give a distinguished email address to agents that send mail
 With Designer
 With correct fields (for MailSend or agents that creates document directly
into mailboxes)
 This way you can use Domino logs to distinguish between agents and
end-users.

• Conclusion
 Ask the developpers
 To give mail agents a mail account that can be tracked easily
 To use From/Principal and ReplyTo fields correctly into the code
 To user agentlog or any log facility when sending mail
o NOT in log.nsf nor Domino Server Console please!
 Manage Domino scheduled agents and log the information

Use Case
• We want to take care of external users

External Users
• Concern is: external users to your company that connect to the
Domino server(s) and use Domino mail functionality
• How many external users use the Domino mail service internally?
• Domino Passthru
 Activity Logging option

Other considerations
• Are my HTTP passwords strong enough to protect current and future
information?

Tips
• Check HTTP password weakness
 Use Domino HTTP stack
 Use DIIOP (Domino Corba program)
 Develop a JAVA agent with NotesFactory
 Loop with Domino users
 Use default passwords
 Use a dictionary
 Store results in an IBM Notes database

Use Case
• How many mails are sent from my mobile users?

Smartphones vs IBM Notes Addicts

• Concern is: Get the Traveler users addicts
• Metrics:
 Number of active users
 Number of mails from the handhelds

• Traveler Users
 Tell traveler users enabled
 Tell traveler users disabled
• Deeper statistics
 What is activated
 Increase the Log level first

• What is activated
 Tell traveler log show
 Tell traveler log handler
 What is my level
 Which handler can I check?
 Handlers
 Tell traveler log handler logNsf level finest
 Tell traveler log handler audit level ALL

• How to activate user info trace:
 Rename the NTSLoggingTemplate.properties in NTSLogging.properties
 Located in <DominoDataDir>/traveler/cfg
 Define LogLevel=ALL
 Define Privacy according to your needs, default is 0x001E (shows everything except BODY
and Attachments) everything is 0x8000
 OR Tell traveler log fields *
 And without any parameter to stop -> “Tell Traveler log fields”
 Restart task Traveler for activating or stopping the logging

• info trace results are located:
 <DominoDataDirectory>/IBM_TECHNICAL_SUPPORT/traveler/logs/xml
 mmdd-hhmm-XXXXX_Sync_<CollectionId>_<SyncKey>_out_200.xml
 mmdd-hhmm-XXXXX_SmartReply_<NOTEUNID>_in.xml
 mmdd-hhmm-XXXXX_SmartForward_<NOTEUNID>_in.xml
 mmdd-hh-mm_Ping_in.xml
 mmdd-hh-mm_Ping_out_200.xml

• The simpler the better
 How to track Traveler users
 The $Mailer field contains the “Traveler” string
 Mail journaling:
 define the correct rule!
o Internal users and all documents
o Beware of the size of the resulting database!
 Do not encrypt the fields you do need for analysis
o $Mailer

• Create a view in mail journaling template
 From an existing view (Person)
 Add a column for $Mailer field
 Add totals

• Conclusions
 Number of mails sent from a device
 Nothing out of the box with Traveler logs
o Need to analyze the log files in xml directory for IBM Lotus Traveler
 Mail Journaling
o Need really few design enhancements
 Need third party software
o Access to the database (Non-HA and HA)
o Live and simple information: audit4mail from move4ideas

Use Case
• We want to get rid of non enterprise data, mainly based on
attachments not on content.

Attachments
• Concern: an excellent starting point to start using IBM Files
 Remove files from mail files to the repository they deserve
 Company information has to be at the company level
• DAOS is not a concern for metrics
 DAOS obfuscate file location (in or out NSF files)
 Quota apply to logical size NOT the physical size
 After migration, DAOS will need to be pruned

Attachments
• A basic and efficient approach for a global picture
 Reset: set stat <stat name>
 Show stat mail.* (or IBM Administrator interface)
 TotalKBDelivered
 TotalKBTransferred
 TotalKBTransferred.SMTP
 DeliveredSize.*

Attachments
• Mail journaling
 Define the rule for internal users only
 Already available in IBM Notes mail journaling databases
 Beware of mail journaling databases size
 You can customize the view for more information
 Extensions of attachments

Attachments
• Use Event task
 Create an event generator for:
 Message transferred to
 Message delivered to

Attachments
• (Mis)Use DAOSEST
 Download version from IBM web site
 Currently 1,7
 Use –v option
 Beware: on really huge architecture it can never end.
 Solution DAOSEST_BUCKETS in notes.ini
o DAOSEST_BUCKETS=16,64,128,256,512,768,1024,2048,3087,4096
o Any value of 10k can be set

Use Case
• We want to know how many huge databases we do have and who
owns such databases.

Huge Databases
• Concern: not a target concern but the source concern
 One mail file of 51k documents (31Gb)
 16 hours to migrate to SmartCloud in the past (non DAOS)
• At which size do we consider a database is huge?
 Time to replicate
 Time to compact (even with option –#)
 Views indexes size (Welcome Domino 9.2!)
 Daosified or non daosified NSF databases

Huge Databases
• The main problems are
 Internal network bandwith
 Slow disks (yes it still exists!)
 Overloaded Domino servers
 No or bad scheduled FUC tasks
o Fixup
o Updall
o Compact
 The notesenv folder for the Onboarding Manager requires space

Huge Databases
• Main problems (continued)
 Calendar
 No ‘Calendar cleanup’ execution on a daily/weekly basis
 Orphan calendar entries with attachments in iNotes
o Cancelled reservations are kept within mail files in iNotes
 Track attachments
 Enterprise or personal
 Track view indexes
 Track databases size

Huge Databases
• Create replica databases with cluster_admin_on
 Faster than any other solution (DAOS or NOT)
 cl copy srv!!database.nsf TransfertServer!!database.nsf REPLICA
• Archive or not Archive?
 A real question
 Archive Pro
 Get rid of inefficient data
 Archive Con
 Aggregate them back

Huge Databases
• How to track the huge database
 IBM Administrator: copy and paste the Files view into a spreadheet Software
 Log.nsf: View Usage/By Size

Use Case
• We want to check for sensitive data and take a decision to migrate that
information or not.

Sensitive Data
• Attachments vs content
 Attachments extensions can help BUT can be obfuscated
 Unions vs VIPs vs C-Levels
• Forwards of sensitive data
• Blind copy of sensitive data
• Encryption
 Are my users do use encryption for sensitive data when mailing the information to
internal or external users?

Sensitive Data
• If forwards matter
 Use the Event task
 Message forwarded to xxx
 Message forwarded to xxx from yyy
• Mail Journaling
 Activate the mail journaling function

Other Considerations
• I want more information on Traveler data than the current one I do
have
• What about user data
 Archives
 User mail rules
 Number of contacts

Tips
• Traveler (for the non HA server)
 Just try: te traveler sql select * from sys.systables
• User mail file content (LotusScript or Java agent or …)
 Loop through all mailfiles
 Get number of archives and their names
 Get number of mail rules
 Get number of contacts (private or not)
 Export information to a notes database
 Export information to a CSV file

Use Case
• We want to migrate people from one region or ‘site’ together.

The Where Dilemma
• Concern:
 Think of collaboration teams
 Think of really bad networks
 New-Caledonia is an excellent example
• What to catch:
 IP addresses from the sender
 Cross check with relationships

IP Address
• IP Address can help define people location
 Log_Sessions=2 in Domino notes.ini
 23/11/2015 10:01:09 Opened session for 'CN=Jérôme Deniau/O=Move4Ideas' (Release
9.0) SessId 520221FB Ip '109.8.206.35' 'Auth' 'C'
 Inbound SMTP only: Mail Journaling (again)
 Do not encrypt the “Received” field
 Create a view that displays the “Received” field(s)
 Analyze the field content

IP Address and Cross Checking
• Cross check log sessions and mail statistics
 Mail statistics: Transferred vs Delivered
 TotalRouted: means number of recipients
 Transferred: means number of physical message transferred
 Delivered: means number of physical messages delivered
 Use Event task

Architecture
• Use of Domino Server task
 MAPS
 Can help in discovering the mail routing between servers
 No interest if only one Domino cluster manages all users

Use Case
• We want to define the best time range to migrate a population.

When
• Concern:
 Avoid user dissatisfaction
• Need to define the best time for switching users
• Communication is essential
• Metrics
 Time when user is not sending mail
 Problem with smartphone users
o Always on line
 Europeans policy on mail during the week-end

When
• Metrics can help but
 Solution is more enterprise policy defined
 Once population are defined
 You can force the switch time and date
 You can inform the end user for the correct procedures.
 You can start replication and onboarding process during off-peak hours
• So what about metrics then

When
• Determine number of mails exchanged during your off-hours
 Problematic with worldwide companies
 Track internal sender and define the best time range for switching
 Depends on replication
 Depends on mail availability on the other side.
 Use Domino mail journaling features and to an analyze on off hours/off-
days for internal senders

Session Conclusion
• Metrics are the starting point
• Metrics help you understand how people work within your company
• This session gives you the main axis to start
• Do not forget to compare the metrics from the source platform to the
targeted system after the migration.

Thank you for listening!
jerome.deniau@move4ideas.com

Migration from IBM Domino to IBM Verse

Recommended

More Related Content

What's hot (20)

Viewers also liked (16)

Similar to Migration from IBM Domino to IBM Verse (20)

Recently uploaded (20)

Migration from IBM Domino to IBM Verse