Where are your directory pain-points? It can be time consuming to configure, deploy and maintain a corporate directory infrastructure. In this session we'll cover the new Lotus Domino 8 directory features that will enable you to accomplish these tasks. We'll highlight Directory Lint, the new verification tool that enables admins to check directory integrity and suggest corrections. By popular demand, Directory Assistance now guides you through LDAP connection configuration and we'll show you how. Is your Lotus Domino LDAP server performance suffering? New LDAP statistics identify slow performing search patterns that your applications are sending. Last but not least, we'll touch on how tracing can help you better troubleshoot the root cause of an issue.
http://kenlin.com
IAC 2024 - IA Fast Track to Search Focused AI Solutions
Lotusphere 2007: ID204 - Take Control of Your IBM Lotus Domino Directory Infrastructure with Lotus Domino 8!
1.
2. ®
ID204: Take Control of Your IBM Lotus Domino
Directory Infrastructure with Lotus Domino 8!
Josh Burchard
IBM Software Group
Domino Directory Team
Ken Lin
IBM Software Group
Domino Directory Team
5. Improved NameLookup Logging: Finer Granularity
NAMELookup logging has been streamlined:
debug_namelookup=1: will continue to supply information as it always has
From the console: set config debug_namelookup=1
NAMELookup::<Lookup> PID:TID ( 42C: 7B) start of routine
NAMELookup::<lookup> Searching name='Terri' (1 of 1 names).
NAMELookup::<lookup> Searching view='$Users' (1 of 1 views).
NAMELookup::<lookup> Searching DBIndex=1.
NAMELookup::<lookup> from cache took 0 msecs
NAMELookup::<lookup> NumReturned=1, TotalNumReturned=1 match(es) for name='Terri'
NAMELookup::<NextNameDatabase> DAResolveDomain found 2 directories: TESTDIR1,NEWDIR2.
NAMELookup::<NextNameDatabase> looking for directory TESTDIR1 in OPEN_NAME_COLLECTION
queue for NRPC Clients.
NAMELookup::<NextNameDatabase> Found directory TESTDIR1 in OPEN_NAME_COLLECTION queue,
DBIndex=2.
NAMELookup::<NAMELookUpDiskLookup> name='Terri' was found '1' match(es) in domain='TESTDIR1'
NAMELookup::<lookup> NumReturned=1, TotalNumReturned=1 match(es) for name='Terri'
NAMELookup::<lookup> DBIndex=1 specified, search is over!
debug_namelookup=2: “Search mode”. Less verbosity
7. ®
Directory Lint (AKA DirLint)
Problems with directory integrity can be hard to diagnose and remedy
8. Background: “Directory Lint” - What a weird name
C/C++ programmers can probably nap through this slide
“Lint” is commonly known as a program that can verify the integrity
of C code by:
Flagging suspicious elements that some pre-configured logic thinks may turn out
to be bugs
“Lint” Itself came from, “the name of the undesirable bits of fiber and fluff found
in sheep's wool”
“IBM Lotus Domino Directory Name Fixer-Upper” wasn't too catchy
Lint programming tool. (2006, November 13). In Wikipedia, The Free Encyclopedia. Retrieved 15:55, December 21, 2006, from
http://en.wikipedia.org/w/index.php?title=Lint_programming_tool&oldid=87512453
10. Overview: Directory Lint
A tool that can be used to provide you with Domino directory
integrity
Reports inconsistencies in Domino directory naming hierarchy
Gives a heads-up about invalid syntax in Domino directory names
that can vex search and login attempts
Scans group member lists to ensure each member exists in an
available Directory Assistance configured directory
8.0’s DirLint is just the beginning! More exciting stuff to come in
future revs!
12. DirLint: The basic flow… straightforward.
You specify one or more Domino directory databases to scan
DirLint runs tests against the given directories
An XML report is generated that flags possible issues
13. Hold on a second!
Q: I know there’s this thing in Domino called Domino Domain Monitoring
(DDM) that flags issues… so why an XML report?
A1: We wanted to roll out this first rev of DirLint and get it in your hands
as soon as possible
A2: Don’t fret! While it might not be in this revision, DDM integration is
coming down the pike!
Oh, all that and we’ll get you started using the XML report by making an
XSLT tool available for you online
Now, back to what DirLint actually does
14. Scan Directory Hierarchy
Using the Domino Registration Process will keep your directory crisp
and clean
Also, adding new entries to Domino through LDAP is safe
BUT! Notes client, Registration-bypassing, name adds may leave
hierarchy gaps
For example:
You add “cn=Jane Dough/ou=OurOrganizationalUnit/o=IBM”
You didn't add a document for “ou=OurOrganizationalUnit”... not such a big deal
in Domino
However, searches in LDAP may fail
Directory Lint will report these types of errors and let you choose
what to fix
15. Sounds a lot like VerifyDIT, to me
You caught me!
VerifyDIT was extended to work with DirLint and:
Be a kinder, gentler incarnation
Report changes, not just arbitrarily modify your directory
Now, you can SEE what will happen if you run the classic
VerifyDIT on your directory BEFORE changes are made
You still have the choice of running the classic VerifyDIT
whenever you want
16. OK, what else? Invalid DN Syntax
Again, using Domino Registration (it’s a great tool) you shouldn’t
need to worry
BUT special “escaped” characters can creep into your directory names
in multiple ways:
Special LDAP chars added through Notes
Example: You were thinking LDAP-style (comma delimited) while typing in:
cn=Josh Burchard,o=IBM
– You really wanted: “cn=Josh Burchard/o=IBM” in Domino
– You get: “cn=Josh Burchard,o=IBM/o=MYDOMAIN”
– Everything, including commas is your entire CN!
17. Invalid DN Syntax
Names added via Domino LDAP before 7.0
Example using the special ‘+’ character:
– The LDAP DN CN=This+That,OU=West,O=Acme should be converted
to Notes DN CN=This"+"That/OU=West/O=Acme.
– However, previous revisions did not correctly escape the + (plus)
character with double-quotes, resulting in a Notes DN
(CN=This+That/OU=Westford/O=Acme) that appears to have a
multi-valued RDN.
– Oops!
Custom programs that bypass syntax checking and write directly to a directory
database
18. Special Characters – Risky Business?
Our translation routines can only be so clever, and special chars that
sneak into the Domino directory may not translate to LDAP the way
you expect and vice versa
Can cause problems when searching for names
Can cause problems when trying to log in with an LDAP-style name
to use a Domino web resource
19. Special Characters – The Li’l Translation List
The following characters need special handling when present in an LDAP DN
less than character <
greater than character >
semicolon character ;
comma character , (within a name, not being used as separator)
plus sign character +
double quote character “
backslash character
equal sign =
A space or # character occurring at the beginning of the string
A space character occurring at the end of the string
Find more about this general topic here:
Domino 7.0 Release notes
http://www-12.lotus.com/ldd/doc/domino_notes/7.0/readme.nsf
Navigate to: Domino Server->About this release->New in this release->New enhancements->LDAP special characterhandling
20. Special Characters - How DirLint can Help
Scans the names in your directory to find out if the special chars
from the chart are embedded
Reports them to you and gives you the choice to decide what to
keep as-is and what to change
22. Group Member Craziness
Problems can arise whenever human input is involved - group
membership lists are no exception
Inserting typos in otherwise valid names
Totally invalid and non-existent names
Etc.
But even correctly entered names that exist today may go away
tomorrow!
23. Group Members - What do I do?
Use Domino Registration when removing things that may be group
members, and you'll be ok
Run DirLint!
DirLint will scan your group member lists and ensure names exist in a directory
available through Directory Assistance
25. Cool! How do I get started?
Simple!
Type: “load dirlint -?” at the Domino server's console command line
to get an overview of all the commands, options and tests DirLint
offers to give you control over directory integrity!
28. Directory Assistance / Secondary LDAP Directories
A way for your Notes applications to achieve …
Internet Authentication
Group Authorization
Mail Addressing, etc.
to secondary directories
30. Suggest - Hostname
DNS SRV records
Per RFC 2782
(Active Directory
automatically does
this)
Server’s DNS suffix
31. Suggest - Base DN for Search
Domino LDAP servers
return empty search base,
denoting the root
32. Suggest - Type Of Search Filter
Domino LDAP (8.0) – dominoAccessGroups for group authorization
IBM Directory Server (8.0) – ibm-allGroups for group authorization
Active Directory (7.0/6.5.5) – memberOf for group authorization
35. Review
Simplifies successful DA/LDAP configurations by suggesting and
immediately testing settings
Suggest buttons are great for configuring DA/LDAP connections for
the first time
Verify buttons are great for re-testing existing DA/LDAP connection
configurations
37. Two Step Approach
1. Identify - How do you determine what’s slow?
Previously, set LDAPDEBUG=1 in Notes.ini to see LDAP server traces
Previously, turn on LDAP Activity Logging
Now, see LDAP.Search.Longest Statistics
2. Remedy - How do you improve slow searches?
Adjust the Domino LDAP server
Adjust the LDAP client application
39. LDAPDEBUG=1 Peeks into Domino LDAP Server
01:12:56.00 PM LDAP> ***** Start search request processing *****
01:12:56.00 PM LDAP> Scope: SUBTREE
01:12:56.00 PM LDAP> Dereference Aliases: 0
01:12:56.00 PM LDAP> TimeLimit: 15
01:12:56.00 PM LDAP> SizeLimit: 0
01:12:56.00 PM LDAP> Attributes to return: ALL
01:12:56.00 PM LDAP> Base: o=klint42p
01:12:56.00 PM LDAP> Filter: (|(cn=ken lin)(givenname=ken lin)
(sn=ken lin)(uid=ken lin)(mail=ken lin))
01:12:56.00 PM LDAP> *** Searching in database c:dominodatanames.nsf...
01:12:56.00 PM LDAP> Type of search: View Search
01:12:56.00 PM LDAP> ... Searching view ($LDAPCN) for match on cn = ken lin
01:12:56.01 PM LDAP> ... Searching view ($LDAPG) for match on givenname =
ken lin
01:12:56.01 PM LDAP> ... Searching view ($LDAPS) for match on sn = ken lin
01:12:56.01 PM LDAP> ... Searching view $Users for match on uid = ken lin
01:12:56.01 PM LDAP> ... Searching view $Users for match on mail = ken lin
01:12:56.01 PM LDAP> GetSearchEntry State
01:12:56.01 PM LDAP> Found matching entry, Note ID: 4942
01:12:56.01 PM LDAP> SendSearchEntry, sending entry CN=Ken Lin,O=klint42p
01:12:56.01 PM LDAP> GetSearchEntry State
01:12:56.01 PM LDAP> Search State
01:12:56.01 PM LDAP> Search State
01:12:56.01 PM LDAP> ***** Count of search entries returned (total): 1 *****
01:12:56.01 PM LDAP> Return Result State (Search operation)
01:12:56.01 PM LDAP> StateReturnResult returning resultCode 0 (Success)
40. Approaches
Previous approaches are laborious
1. Turn on LDAPDEBUG=1 Tracing or Activity Logging
2. Restart LDAP server
3. Resend LDAP traffic
4. Analyze lots and lots of data
5. Remedy
6. Repeat steps 2-5 until satisfied
7. Turn off tracing or logging
8. Resume normal LDAP application operation
New LDAP.Search.Longest Domino statistics (since 7.0.2)
1. SHOW STAT LDAP
2. Analyze just a few statistics
3. Remedy
No digging through lots of traces!
No down time!
No recreating LDAP traffic - these stats always maintained!
42. Decoding LDAP.Search.Longest.Pattern
basedn - where to start searching
o=klint42p ? ? sub ? (location=%v) ? timelimit=15
Modeled after part of RFC 4516 - LDAP URL
ldap://host:port/basedn?attributes?scope?filter?extensions
attributes - to return
scope - relative to basedn (base, subtree, onelevel)
filter - %v is user-supplied value
extensions - from client
44. LDAP.Search.Longest Statistics
It is often the search pattern, not every search instance, that
determines the overall efficiency of the Domino LDAP search.
LDAP applications search by reusing a limited set of search
patterns, but with different values.
LDAP applications allow their administrators to customize the
search patterns used.
Directory Assistance – LDAP “Type of search filter to use”
Sametime – stconfig.nsf LDAPServer document’s “search filters”
Portal – wmm.xml configuration file
The new LDAP.Search.Longest Domino statistics reveal the search
patterns ordered by slowest average times.
Since the LDAP server does not have to record tremendous volumes
of individual searches, the LDAP.Search.Longest statistics are
always available and does not require a “debug” mode.
46. How Domino LDAP Server Searches
View Search
For attributes in Pubnames.ntf view indices
Full Text Search
For attributes not in Pubnames.ntf view indices
All Search
For attributes not in Pubnames.ntf view indices when no FT Index present
Visits every document in Domino directory
Specialized Searches
For group membership, modified time, Universal Note ID-based searches, etc.
QR Cached Search
For previously issued searches
47. View Search
01:12:56.00 PM LDAP> ***** Start search request processing *****
01:12:56.00 PM LDAP> Scope: SUBTREE
01:12:56.00 PM LDAP> Dereference Aliases: 0
01:12:56.00 PM LDAP> TimeLimit: 15
01:12:56.00 PM LDAP> SizeLimit: 0
01:12:56.00 PM LDAP> Attributes to return: ALL
01:12:56.00 PM LDAP> Base: o=klint42p
01:12:56.00 PM LDAP> Filter: (|(cn=kenFilter: (|(cn=kenFilter: (|(cn=kenFilter: (|(cn=ken lin)(givennamelin)(givennamelin)(givennamelin)(givenname=ken=ken=ken=ken linlinlinlin))))
(sn=ken(sn=ken(sn=ken(sn=ken lin)(uidlin)(uidlin)(uidlin)(uid=ken=ken=ken=ken lin)(maillin)(maillin)(maillin)(mail=ken=ken=ken=ken linlinlinlin))))))))
01:12:56.00 PM LDAP> *** Searching in database c:dominodatanames.nsf...
01:12:56.00 PM LDAP> Type of search: View SearchType of search: View SearchType of search: View SearchType of search: View Search
01:12:56.00 PM LDAP> ... Searching view ($LDAPCN) for match on cn = ken lin
01:12:56.01 PM LDAP> ... Searching view ($LDAPG) for match on givenname =
ken lin
01:12:56.01 PM LDAP> ... Searching view ($LDAPS) for match on sn = ken lin
01:12:56.01 PM LDAP> ... Searching view $Users for match on uid = ken lin
01:12:56.01 PM LDAP> ... Searching view $Users for match on mail = ken lin
01:12:56.01 PM LDAP> GetSearchEntry State
01:12:56.01 PM LDAP> Found matching entry, Note ID: 4942
01:12:56.01 PM LDAP> SendSearchEntry, sending entry CN=Ken Lin,O=klint42p
01:12:56.01 PM LDAP> GetSearchEntry State
01:12:56.01 PM LDAP> Search State
01:12:56.01 PM LDAP> Search State
01:12:56.01 PM LDAP> ***** Count of search entries returned (total): 1 *****
01:12:56.01 PM LDAP> Return Result State (Search operation)
01:12:56.01 PM LDAP> StateReturnResult returning resultCode 0 (Success)
Simplify!
49. Query Results Cache’d Search
***** Start search request processing *****
Scope: SUBTREE
Dereference Aliases: 0
TimeLimit: 15
SizeLimit: 0
Attributes to return: ALL
Base: o=klint42p
Filter: (|(cn=kenFilter: (|(cn=kenFilter: (|(cn=kenFilter: (|(cn=ken lin)(givennamelin)(givennamelin)(givennamelin)(givenname=ken=ken=ken=ken linlinlinlin))))
(sn=ken(sn=ken(sn=ken(sn=ken lin)(uidlin)(uidlin)(uidlin)(uid=ken=ken=ken=ken lin)(maillin)(maillin)(maillin)(mail=ken=ken=ken=ken linlinlinlin))))))))
Found entry in LDAP QR Cache.Found entry in LDAP QR Cache.Found entry in LDAP QR Cache.Found entry in LDAP QR Cache.
***** Count of search entries returned (total): 1 *****
Return Result State (Search operation)
StateReturnResult returning resultCode 0 (Success)
50. Fallback To All Search
***** Start search request processing *****
Scope: SUBTREE
Dereference Aliases: 0
TimeLimit: 15
SizeLimit: 0
Attributes to return: ALL
Base: o=klint42p
Filter: (location=Filter: (location=Filter: (location=Filter: (location=wchwchwchwch))))
*** Searching in database c:dominodatanames.nsf...
Type of search: FT SearchType of search: FT SearchType of search: FT SearchType of search: FT Search
... No FT index was found... No FT index was found... No FT index was found... No FT index was found
... Fallback to All Search... Fallback to All Search... Fallback to All Search... Fallback to All Search
... Getting entries in ($LDAPRDNHier)
GetSearchEntry State
Found matching entry CN=Ken Lin/O=klint42p (NoteID: 4942)
SendSearchEntry, sending entry CN=Ken Lin,O=klint42p
GetSearchEntry State
Search State
Search State
***** Count of search entries returned (total): 1 *****
Return Result State (Search operation)
StateReturnResult returning resultCode 0 (Success)
LDAP Server: You should full text index Domino directory
names.nsf on klint42p/klint42p to improve search performance
for filters like '(location=x)'
Full Text Index!
52. Full Text Search
***** Start search request processing *****
Scope: SUBTREE
Dereference Aliases: 0
TimeLimit: 15
SizeLimit: 0
Attributes to return: ALL
Base: o=klint42p
Filter: (location=Filter: (location=Filter: (location=Filter: (location=wchwchwchwch))))
*** Searching in database c:dominodatanames.nsf...
Type of search: FT SearchType of search: FT SearchType of search: FT SearchType of search: FT Search
FT Query: ([$$O] Contains ("klint42p")) AND
(([location] Contains ("wch")))
Type of search: Modified Since FT SearchType of search: Modified Since FT SearchType of search: Modified Since FT SearchType of search: Modified Since FT Search
GetSearchEntry State
Found matching entry, Note ID: 4942
SendSearchEntry, sending entry CN=Ken Lin,O=klint42p
GetSearchEntry State
Search State
Search State
***** Count of search entries returned (total): 1 *****
Return Result State (Search operation)
StateReturnResult returning resultCode 0 (Success)
53. Group Membership and dominoAccessGroups
If you see many search patterns like this …
??sub?(&(objectclass=%v)(member=%v))
the application may be attempting to performing many series of
nested group membership searches
e.g., “cn=Ken Lin,ou=Westford,o=IBM” belongs to
“cn=LDAP Server Dev” belongs to
“cn=Iris Directory Team” etc.
For such situations, consider reconfiguring the application to use a
single query to retrieve the person’s new 8.0 dominoAccessGroups
attribute instead
Domino Directory Assistance - LDAP
Type of search filter = Domino LDAP
Portal and Websphere Member Manager (WMM) -based applications
groupMembershipAttributeMap = "dominoAccessGroups:nested"
54. Relative LDAP Search Speeds
QR Cache’d Search
All Search
View Search
Full Text Search
If DDM.nsf shows a Fallback to All Search warning, Full Text Index the specified
Domino directory and make sure the Update task is running.
If application’s LDAP search pattern contains terms that are not indexed view
fields, see if they can either be eliminated or changed to use indexed fields.
If different LDAP applications use equivalent or similiar filters, evaluated if they
can be made identical.
e.g., Technote 1197769 – Change Websphere Portal People Finder wmm XML files from
pluginAttributeName=“displayName” to pluginAttributeName=“cn” for Domino LDAP < 7.0.2
e.g., If one application uses “(|(cn=%v)(givenName=%v)(sn=%v))” and another uses
“(|(cn=%v)(sn=%v)(givenName=%v))”, rearrange one to match the other
55. Miscellaneous
Notes.ini Variables
LDAPMaxLongestSearchCount - Number of sets of statistics maintained
Default is LDAPMaxLongestSearchCount = 20
LDAPMaxLongestSearchCount = 0 turns off collection
LDAPMaxLongestSearchCount = 50 is maximum
In general, too many statistics will slow down Domino
LDAPMinLongestSearchTime - Searches shorter than this milisecond interval are
not collected
Default is LDAPMinLongestSearchTime = 100 (i.e., 0.1 sec)
LDAPMinLongestSearchTime = 0 collects all searches
56. Review
Identify the slowest searches using SHOW STAT LDAP command
Available since 7.0.2!
Target the slowest search patterns that have the highest count
Check the DDM Directory events for Full Text Index recommendations
Remedy performance …
Domino LDAP Server: Full text index Domino directories as necessary
LDAP Application: Tweak the application’s search filters so …
View searches are used
Complexity of the search filter is reduced
– Can you remove terms?
– Can you use dominoAccessGroups for group membership searches?
58. See Also
ID207: IBM Lotus Domino 8 Directory Deployment to Address TCO
SW 3-4, Monday 11:00-12:00
8.0 directory features
Directory roadmap
BOF305: IBM Lotus Domino Directory Integration
SW Macaw 1-2, Wednesday 5:45-6:45
Directory roadmap
Open discussion
L101: Meet the Developers Lab
DL Asia 1-2
L105: Deployments, Performance and Interoperability
DL Europe 3-4
Google “Domino Directory FAQ”
We monitor “Notes/Domino 6 and 7 Forum” and
“Business Partner Forum”