BP101 - 10 Things to Consider when Developing & Deploying Applications in Large Scale Environments

10 Things to Consider when Developing & Deploying Applications in Large Scale Environments Martijn de Jong | ilionx BP101

Agenda Speaker Introduction Examples of Large Scale Environment Applications Characteristics of a Large Scale Environment 10 Things to Consider... Wrap-up Q & A

Speaker Introduction So who am I? M.Sc. Electrical Engineering at the University of Delft, The Netherlands Psychology & Ergonomics at the University of Stellenbosch, South Africa Advanced Certified IBM Lotus® Notes® & Domino® 8 Application Developer & System Administrator and a Certified Lotus Instructor 2000: Lotus Notes trainer at IBM 2002: Started with Domino Development yet moved to System Administration job 2004: Lotus Consultant at ilionx 2006: Application Developer for the ABN AMRO Bank Messaging & Groupware Architecture Team 500+ Domino servers 135.000+ Notes users Single domain I blog at http://www.socialsoftwareblog.nl Contact me at: mdejong@ilionx.com / martdj@martdj.nl

Examples of Large Scale Environment Applications System to maintain & shape the Mail Infrastructure User's Mail files Domino Directory HRM Systems Information Store Users Admins Messaging Portal Mail file Tooling Mail file Upgrade Tool Server Logs

Examples of Large Scale Environment Applications System for requesting, monitoring & maintaining mobile devices Users Providers Support SQL Database Servers Collector Front Central External Parties Internal Support

Characteristics of a Large Scale Environment Many servers Many users – Large Domino Directory Large number of applications Large amounts of documents in some applications Many daily updates in some applications Strict procedures Separation between Developer & Infrastructure Teams Some connections between servers have limited bandwidth

What does this mean for a developer? Compared to developing for smaller environments, developers have to give extra consideration to: System Administration costs Efficient usage of system resources (disk space, cpu time, memory usage) Network load Security requirements

1. Consider the Deployment Effort Avoid multiple unique copies of the same database for one application Replication Settings don't replicate. If you need them to be different from the default, set them with an agent Example code to set the Deletion Stub Cutoff Period Set repinfo = thisdb.ReplicationInfo ' Grab the preferred deletion stub cutoff date from a profile document CutoffPeriod = Clng(profdoc.GetItemValue("prfDelStubCutoff")(0)) If repinfo.CutoffInterval <> CutoffPeriod Then repinfo.CutoffInterval = CutoffPeriod repinfo.Save End If “ Don't allow simple search” also doesn't replicate Call thisdb.SetOption(76, True) ' sets don't allow simple search

1. Consider the Deployment Effort (2) It's easier to deploy an application on all servers than on 60% of the servers It should be possible to deploy a new version of your application by replacing the design on the administration server. If this is not the case, think of a way to make it so Deploying a replica of your application on a new server should just be a matter of creating a new replica. If any other actions are necessary, consider building extra code to automate them Remember: The more time it takes for the administrators to deploy your application, the more time it will take you to verify their work!

2. Design Flexibility Deploying design changes can be a lengthy process in larger organisations. Minimize the need by keeping your designs as flexible as possible Examples Don't hard code mails in your design. Keep content in profile documents or use special memo documents

2. Design Flexibility Deploying design changes can be a lengthy process in larger organisations. Minimize the need by keeping your designs as flexible as possible Examples Don't hard code mails in your design. Keep content in profile documents or use special memo documents Don't use the Style Sheets design element. Keep your CSS code in documents Run agents on -Any Server- and use profile documents to define the server(s) they should run on

3. How to run your agent where and when you'd want them to run Run agents on “-Any Server-” . Let them run as often as needed Use a profile document for selection Can be expanded by adding time slots for the agents

3. How to run your agent where and when you'd want them to run Code example: Set profdoc = thisdb.GetProfileDocument("prfDb") ' Don't run the agent on servers that it shouldn't run on arrRunServerGroups = profdoc.GetItemValue("prfAgentServerGroups") [..] For i = 0 To Ubound(arrRunServerGroups) If Not Isnull(Arraygetindex(ExpandGroup(arrRunServerGroups(i), False), thisdb.Server)) Then blnServerMatched = True Exit For End If Next ' Exit if it's not in any of the groups If Not blnServerMatched Then Exit Sub ' Check if the server is in the exclude list NotRunServers = profdoc.GetItemValue("prfNonMFIServers") If Isnull(Arraygetindex(NotRunServers, thisdb.Server)) Then ' Next check if the agent should run today Call htoday.SetNow collectdays = profdoc.GetItemValue("prfMTCollectDays") If Not Isnull(Arraygetindex(collectdays,Weekday(htoday.DateOnly))) Then <Your agent code here> End If End If

4. Special Signer IDs Many organisations require a special signer ID for agents that: Run unrestricted methods and operations Send Mail Try to limit the number of these types of agents Example: Let all agents that need to send mail, create the mail and save it. Use one agent to send the mail

5. Do you really need that view? In applications with many documents, views can take up big chunks of disk space and processing power of the server It's better to build a few good user views than many different views which a user might want Using an extra sortable column is better than building an extra view 99% of the time there's no need for a double sortable column For lookup views, describe (in the comment field) which design elements use that view Never use @Now/@Today in a view Alternative: See AD214: What Every Domino Developer Should Know About Performance

6. Do your agents run out of time? How long may your agents run: Agent running in the client: Indefinitely Agent running on the server: Depends on type, trigger & time Before new mail arrives: Runs in the Router Task (Configuration Document)

6. Do your agents run out of time? How long may your agents run: Agent running in the client: Indefinitely Agent running on the server: Depends on type, trigger & time Before new mail arrives: Runs in the Router Task WebQueryOpen / WebQuerySave Form event: Runs in the HTTP Task (Server document - Internet Protocols - Domino Web Engine)

6. Do your agents run out of time? How long may your agents run: Agent running in the client: Indefinitely Agent running on the server: Depends on type, trigger & time Before new mail arrives: Runs in the Router Task WebQueryOpen / WebQuerySave Form event: Runs in the HTTP Task Scheduled / triggered: Runs in the Amgr Task Simple / Formula Agents (Server Document - Basics)

6. Do your agents run out of time? How long may your agents run: Agent running in the client: Indefinitely Agent running on the server: Depends on type, trigger & time Before new mail arrives: Runs in the Router Task WebQueryOpen / WebQuerySave Form event: Runs in the HTTP Task Scheduled / triggered: Runs in the Amgr Task LotusScript / Java Agents (Server Document – Server Tasks – Agent Manager)

6. Do your agents run out of time? How long may your agents run: Agent running in the client: Indefinitely Agent running on the server: Depends on type, trigger & time Before new mail arrives: Runs in the Router Task WebQueryOpen / WebQuerySave Form event: Runs in the HTTP Task Scheduled / triggered: Runs in the Amgr Task tell amgr run “<app name>” '<agent name>' : Runs in a separate Amgr Task Agent can run indefinitely... “ Call notesagent.RunOnServer” : Server Task Agent can run indefinitely...

6. Do your agents run out of time? Five ways to deal with agents running out of time Optimize them AD214: What Every Domino Developer Should Know About Performance Split up your agent For completely separate actions only Divide the workload in parts and run the agent multiple times Use a key field to recognize where the agent was cut off last Run small batches divided on alphabet or DocumentUniqueID Divide the workload over multiple servers Make each server responsible for specific documents based on a field value or another key A combination

7. Do you need all documents in all replicas? Sometimes a document is only relevant for a specific region, cluster or even a single server Examples The Administration Requests database The Database Catalog If your applications contains large quantities of these documents consider... Replication Formulas

7. Do you need all documents in all replicas? (2) Any Server – Any Server replication formula has some problems (easily gets overruled by a server specific formula) Better to use <Server Name> – Any Server replication formulas Set these replication formulas with an agent (consider deployment effort) Use @UserName for the current server Disable replication of Replication Formulas Deletion stubs, profile documents & agent data documents always replicate Watch out for Domino 6.0.3 / 6.5.0 when using replication formulas

7. Do you need all documents in all replicas? (3) Example of agent to set Replication Formula CreateRepFormula = True Set repinfo = thisdb.ReplicationInfo ' Check if a proper replication formula entry exists Set repentry = repinfo.GetEntry("-", thisdb.Server, False) If Not repentry Is Nothing Then If repentry.Views = "" Then If repentry.IsIncludeDocuments Then If repentry.Formula = | @IsMember(@UserName;Servers)| Then If repentry.IsIncludeACL Then If repentry.IsIncludeAgents Then If repentry.IsIncludeForms Then If Not repentry.IsIncludeFormulas Then CreateRepFormula = False End If [..] ' If it doesn't, create/modify it If CreateRepFormula Then Set repentry = repinfo.GetEntry("-", thisdb.Server, True) repentry.IsIncludeACL = True repentry.IsIncludeAgents = True repentry.IsIncludeDocuments = True repentry.IsIncludeForms = True repentry.IsIncludeFormulas = False repentry.Views = "" repentry.Formula = | @IsMember(@UserName;Servers)| repentry.Save End If

8. Beware of deletion stubs How do deletion stubs expire: For default settings (90 days) deletion stubs are removed once every 30 days On 'expiry day' all deletion stubs older than 90 days are removed

8. Beware of deletion stubs (2) You know you have a problem with deletion stubs when: Creating a new copy of your database takes less than a minute and 5 MB of disk space Creating a new replica of your database takes over an hour and 125 MB of disk space Cause: I copy-pasted code from another application which weekly removed all documents and recreated them instead of updating them

8. Beware of deletion stubs (3) In case your documents have fields that need to be removed for certain cases, this code works: Sub EmptyDocument(doc As NotesDocument) Dim Allitems As Variant Dim i As Integer Allitems = doc.Items If Not Isempty(Allitems) Then For i = 0 To Ubound(Allitems) ' reason for the if then are docs with multiple fields of the same name If Not Allitems(i) Is Nothing Then Call Allitems(i).Remove Next End If End Sub (Only for extreme cases)

9. Replication vs Mail vs Data Copy Agent To process documents on a specific server you can use either mail, replication or a Data Copy Agent to get it there

10. Try to limit your replication traffic Replication Formulas can help Don't save documents unless you have to Set doc = vw.GetFirstDocument While Not doc Is Nothing Set ndoc = vw.GetNextDocument(doc) ' Pick out the mail documents. Leave the rest for the process action agent If doc.GetItemValue("Form")(0) = "Memo" Then Call doc.ReplaceItemValue("Form", "Action") Else If doc.GetItemValue("Form")(0) = "Reply" Then Call SendRejectMail(doc.GetItemValue("From")(0)) Call doc.ReplaceItemValue("Processed", "1") End If End If Call doc.Save(False, False) Set doc = ndoc Wend Bad

10. Try to limit your replication traffic Replication Formulas can help Don't save documents unless you have to Dim ShouldSave As Boolean Set doc = vw.GetFirstDocument While Not doc Is Nothing Set ndoc = vw.GetNextDocument(doc) ShouldSave = False ' Pick out the mail documents. Leave the rest for the process action agent If doc.GetItemValue("Form")(0) = "Memo" Then Call doc.ReplaceItemValue("Form", "Action") ShouldSave = True Else If doc.GetItemValue("Form")(0) = "Reply" Then Call SendRejectMail(doc.GetItemValue("From")(0)) Call doc.ReplaceItemValue("Processed", "1") ShouldSave = True End If End If If ShouldSave Then Call doc.Save(False, False) Set doc = ndoc Wend

10. Try to limit your replication traffic (2) Don't flag documents as a workaround for @Today view selection formulas During the design phase of your application, try to make a good estimate of the expected number of daily document additions/updates of each type of document in your application

Wrap-Up (2) When administrators and developers cooperate they can make better performing applications Recommended Presentations AD214 - What Every Domino Developer Should Know About Performance AD210 - Built to Last: IBM Lotus Domino Development Strategies BP102 - A New Approach to Internationalizing IBM Lotus Domino Applications Code example database can be downloaded from: http://www.martdj.nl/martijn/bp101.zip You can contact me at: mdejong@ilionx.com / martdj@martdj.nl

Thank You! Please remember the session survey

Legal disclaimer © IBM Corporation 2008. All Rights Reserved. The information contained in this publication is provided for informational purposes only. While efforts were made to verify the completeness and accuracy of the information contained in this publication, it is provided AS IS without warranty of any kind, express or implied. In addition, this information is based on IBM’s current product plans and strategy, which are subject to change by IBM without notice. IBM shall not be responsible for any damages arising out of the use of, or otherwise related to, this publication or any other materials. Nothing contained in this publication is intended to, nor shall have the effect of, creating any warranties or representations from IBM or its suppliers or licensors, or altering the terms and conditions of the applicable license agreement governing the use of IBM software. References in this presentation to IBM products, programs, or services do not imply that they will be available in all countries in which IBM operates. Product release dates and/or capabilities referenced in this presentation may change at any time at IBM’s sole discretion based on market opportunities or other factors, and are not intended to be a commitment to future product or feature availability in any way. Nothing contained in these materials is intended to, nor shall have the effect of, stating or implying that any activities undertaken by you will result in any specific sales, revenue growth or other results. All customer examples described are presented as illustrations of how those customers have used IBM products and the results they may have achieved. Actual environmental costs and performance characteristics may vary by customer. IBM, the IBM logo, Lotus, Lotus Notes, Notes, Domino and Lotusphere are trademarks of International Business Machines Corporation in the United States, other countries, or both. Java and all Java-based trademarks are trademarks of Sun Microsystems, Inc. in the United States, other countries, or both.

BP101 - 10 Things to Consider when Developing & Deploying Applications in Large Scale Environments

Recommended

More Related Content

Similar to BP101 - 10 Things to Consider when Developing & Deploying Applications in Large Scale Environments (20)

More from Martijn de Jong (7)

Recently uploaded (20)

BP101 - 10 Things to Consider when Developing & Deploying Applications in Large Scale Environments