3. Previously, on days of our lives… In Schema 101, we covered: Schema and database concepts Viewing your schema Editing your schema Schema tools A walkthrough of customizing your schema using SchemaScript Assorted tips and tricks for working with your schema Slides and recording: http://dovetal.es/schema101webinar
4. Topics SQL Views MTM Tables Role Objects Pseudo Relations Exclusive Relations Change Fields Closer look at ADP Tables Q&A
6. SQL Views Introduced in Clarify 11.5 Allows for explicitly defining the SQL for a view, as opposed to the schema tool dynamically building the SQL
7. Why would we use a SQL View? A view that uses aggregates (max, min, count, sum, avg, etc.). A view using multiple, unrelated tables. A view that uses the UNION keyword to consolidate between multiple tables or views. A view that accesses linked tables, or other databases.
8. SQL View elements Name Id List of columns Each column has its own attributes (name, datatype, length, etc.) Sql Database specific sql (if on Clarify 13.1+) Looks more like a table, than a view
9. Lets look at a SQL View in Dovetail SchemaEditor format
10. <sqlView name=“dt_contact_view" id="4756" > <sql><![CDATA[ SELECT DISTINCT r.objid as contact_role_objid, c.objid as contact_objid, s.objid as site_objid, c.first_name, c.last_name, c.phone, c.update_stamp as last_modified, s.name as site_name, s.site_id as site_id, r.primary_site as is_primary_site, c.status as is_active, c.e_mail, r.role_name FROMtable_contact c INNER JOIN table_contact_role r ON c.objid = r.contact_role2contact INNER JOIN table_site s ON s.objid = r.contact_role2site WHEREc.status = 0 ]]></sql> <sqlViewColumn name="contact_role_objid" dataType="Integer" /> <sqlViewColumn name="contact_objid" dataType="Integer" /> <sqlViewColumn name="site_objid" dataType="Integer" /> <sqlViewColumn name="first_name" dataType="String" /> <sqlViewColumn name="last_name" dataType="String" /> <sqlViewColumn name="phone" dataType="String" /> <sqlViewColumn name="last_modified" dataType="DateTime" /> <sqlViewColumn name="site_name" dataType="String" /> <sqlViewColumn name="site_id" dataType="String" /> <sqlViewColumn name="is_primary_site" dataType="Integer" /> <sqlViewColumn name="is_active" dataType="Integer" /> <sqlViewColumn name="e_mail" dataType="String" /> <sqlViewColumn name="role_name" dataType="String" /> </sqlView>
12. VIEW fc_contact_view 4756 SQL="SELECT DISTINCT c.objid as contact_objid, s.objid as site_objid, c.first_name, c.last_name, c.phone, c.update_stamp as last_modified, s.name as site_name, s.site_id as site_id, r.primary_site as is_primary_site, c.status as is_active, c.e_mail FROM table_contact c INNER JOIN table_contact_role r ON c.objid = r.contact_role2contact INNER JOIN table_site s ON s.objid = r.contact_role2site WHERE c.status = 0“ FIELDS contact_objid CMN_DATA_TYPE="long integer" DB_DATA_TYPE=0 OPTIONAL USER_DEFINED COMMENT="" , site_objid CMN_DATA_TYPE="long integer" DB_DATA_TYPE=0 OPTIONAL USER_DEFINED COMMENT="" , first_name CMN_DATA_TYPE="char pointer" DB_DATA_TYPE=7 OPTIONAL USER_DEFINED COMMENT="" … FIELDS_END VIEW_END;
13. SQL View in BOLT http://localhost/bolt/sql_view.asp?type_id=4756&type_name=fc_contact_view
14. SQL View Tip: Column Ordering The order of the SQL view columns must be the same in the SQL as in the list of SQL view columns. For example, if your SQL select statement is select first_name, last_name, favorite_color..., then the list of columns should be in the same order, i.e. first_name is defined first, followed by last_name, and then favorite_color.
15. SQL View Tip: S_ Columns on Oracle If a SQL view column is defined as searchable, then the SQL statement should include the S_ column immediately after the non S_ column. The list of SQL view columns should include the normal column as well as the S_ column.
17. SQL View – another example Query for all of the cases that have any open subcases SQL View thatincludes case data, plus a column that indicates “how many open subcases does this case have” http://dovetal.es/sqlView
20. Many-To-Many Relations What are they? MTM relations allow multiple instances in the first object to be associated with multiple instances in the second object, and multiple instances in the second object to be associated with multiple instances in the first object.
21. More than one contract can be related to the same site . . . id type 17101-2011 Hardware parts 85274-2010 Software Upgrade 98741-2011 Service Warranty site_id name 12345 Dovetail site contract . . . and more than one site can be related to the same contract site_id name 12345Dovetail – Austin HQ 45678Dovetail – Dallas 10011Dovetail – Boston id type 17101-2011 Hardware Parts contract site
22. MTM schema MTM relations are implemented with an MTM table. The MTM table consists of two columns, each containing related objids. One column is named for the MTM relation from one object to the second. The second column is named for the inverse relation from the second object back to the first
23. contract2site (MTM) contract site cust_loc2contract (MTM) contract2 cust_loc2 site contract 29360129 29360130 29360129 29360131 29360130 29360129 29360131 29360131 29360131 29360129 29360131 29360130 id objid COMP-1024 29360129 PRNT-9458 29360130 SRVC-5840 29360131 objid name 29360129 NanoComp 29360130 MicroTech 29360131 PicoProc site contract mtm_site5_contract0
24. Column names are backwards from what you would expect contract2 cust_loc2 site contract 29360129 29360130 29360129 29360131 29360130 29360129 29360131 29360131 29360131 29360129 29360131 29360130 id objid COMP-1024 29360129 PRNT-9458 29360130 SRVC-5840 29360131 objid name 29360129 NanoComp 29360130 MicroTech 29360131 PicoProc site object instances contract object instances mtm_site5_contract0 The contract2site column holds objids of contracts The cust_loc2contract holds objids of sites
25. Creating a MTM table You don’t. Simply create a relation using your schema editing tool of choice (ddcomp, DD Editor, Dovetail SchemaEditor, etc.) The tool will automatically create the MTM table for you
26. MTM Tables in the Schema MTM tables are NOT objects in the Clarify sense They do not show up in your schema file or in your Data Dictionary document BOLT does tell you the MTM table namehttp://localhost/bolt/table.asp?type_id=86&type_name=contract MTM table name is useful when doing direct SQL, or reporting
27. MTM Table Name The name of each MTM table is based on the related objects: mtm_<object1><number1>_<object2><number2>where the object names are the related objects; the number values represen the positional value of that relation in the schema metadata.
28. MTM Table Name: site2contract On table site, the MTM relation to contract is the 5th relation. (using zero based index counting) On table contract, the MTM relation to site is the 0th relation. (using zero based index counting) Table site has a table id of 52. Table contract has a table id of 86. 52 < 86So, site comes before contract. Which leads us to: mtm_site5_contract0
31. Role Tables A common schema pattern is a role table. These objects are similar to MTM tables, in that they implement an MTM relationship between two objects, but the relationship has attributes, so we have an object with fields that contain data. Role objects have two MTO relations.
32. Role Table Example contact and site are related using the contact_role table. role_name indicates the contact’s role at the site. primary_site identifies the normal location of the contact. contact_role site contact contact2contact_role contact_role2site role_name primary_site
35. Pseudo Relations Implements a relation using two fields. focus_type -- identifies the object (by type_id) for the relation. focus_lowid -- identifies the specific instance (objid) of the object. Commonly run across in time_bomb, act_entry
36. Pseudo Relations Can’t be used in traversals Can’t use RelateRecords type methods (ClearBasic, Dovetail SDK) Because they’re not real relations
37. Pseudo Relation Example When creating a time bomb for a case: In table_time_bomb set focus_type = 0 (the type id of the case table) set focus_lowid = <the objid of the actual case>
40. Exclusive Relations Exclusive Relations are Pseudo-Relations improved. Still have 2 fields: One for the type_id of the related object One for the objid of the related row Can be used in traversals Can be used in RelateRecords type methods
41. Exclusive Relations Exclusive Relation essentially means that only one out of a set of relations will be populated. It’s used to reduce the number of physical columns that need to be created (when most of those columns would be sparsely populated)
42. Exclusive Relations Example A single routing request (r_rqst) will only ever be related to one of these at any one time:bug, case, subcase, task So all of those relations will go into a “set” Only one of those will be filled at one time.
43. Exclusive Relations Example On the r_rqst table, we define a exclusive relation set. In this set: TYPE_FIELDwhich field will store the table id of the related row OBJID_FIELD which field will store the objid of the related row List of relationsList of relations that belong to this set
44. Exclusive Relations Example OBJECT r_rqst 5378 FIELDS … source_type CMN_DATA_TYPE="long integer" DB_DATA_TYPE=1, source_lowid CMN_DATA_TYPE="long integer" DB_DATA_TYPE=0, … FIELDS_END RELATIONS … , EXCLUSIVE_SET route_for TYPE_FIELD = source_type OBJID_FIELD = source_lowid r_rqst2case MTO case INV_REL=case2r_rqst, r_rqst2bug MTO bug INV_REL=bug2r_rqst, r_rqst2subcase MTO subcase INV_REL=subcase2r_rqst EXCLUSIVE_SET_END RELATIONS_END OBJECT_END;
45. Exclusive Relations I rarely use exclusive relations But, it’s still nice to know that they’re available if you need them.
46. Exclusive Relations Lets look at the r_rqst table in BOLT And lets look at its physical definition:
49. Change Fields When this field changes, update this other date field automatically. First field is marked as CHANGE_FIELD Second field is marked as CHANGE_DATE
50. Change Fields Works in any POMS-based application (Clarify Client, ClearBasic, etc.) Works in Dovetail SDK based applications
51. Change Fields CHANGE_FIELD indicates: this is a field that I want to know about changes to CHANGE_DATE indicates: when a change_field event occurs, update this timestamp
52. Change Fields Each database table can have multiple CHANGE_FIELDsbut only one CHANGE_DATE
53. Change Fields - Example OBJECT rsrc 5380 COMMENT="Represents availability of a resource, either queue or user, for assignment of routing requests" FIELDS … last_update CMN_DATA_TYPE="clk_ty_time" DB_DATA_TYPE=9 CHANGE_DATE , rsrc_state CMN_DATA_TYPE="long integer" DB_DATA_TYPE=1 CHANGE_FIELD DEFAULT="0" FIELDS_END
58. ADP (ADaPter) Layer Metadata defining the Clarify schema Set of tables in the Clarify database which contain “data about the containers of data” These table names all start with adp_
59. Database structures and ADP data must remain in sync! This is why we use schema editing tools.
63. Flags & Other ADP Data Each ADP table has a flags column that stores additional information. These flags are difficult to figure out. I don’t recommend you try. BOLT and schema files will provide the info Don’t muck with them manually! Use a proper schema editing tool. Bad ADP data can cause havoc in your system. Don’t muck with them manually!
64. ADP Complexities, an example Relations are stored in ADP_SCH_REL_INFO Each relation has a type (rel_type) Simple, right? But… 10 and 11 are used if the relation field is used in a schema index. Change a schema index, and the rel_type changes
65. ADP Complexities, another example A view can be “optimized” When the view is optimized, schema tools determine whether the join actually needs to be executed or not -- based on objid location. During optimization a second set of rows is created in ADP_VIEW_JOIN_INFO for each view with the executed joins.
66. ADP Complexities I point out these complexities as a reminder: Please don’t modify the ADP Tables yourself! Use a proper schema editing tool. This will make things easier for you, your co-workers, and those who follow in your footsteps.
68. ADP_TBL_OID Used for objid generation 1 row for each table in the system Only 2 columns: type_id and obj_num Obj_num is the next offset used for objid Basic algorithm for objid generation: objid = (2^28 * adp_db_header.site_id) + (next_val + 1)
69. ADP_TBL_OID Not used on older Clarify versions on Oracle. next_val comes from an Oracle sequence instead of adp_tbl_oid. Same basic algorithm: objid = (2^28 * adp_db_header.site_id) + (next_val + 1)
70. ADP_TBL_OID_UNUSED Applications can grab a bunch of objids at one time. Any unused ones are returned to the ADP_TBL_OID_UNUSED table. Objid = (2^28 * adp_db_header.site_id) + (next_val + 1) next_val is obtained first from adp_tbl_oid_unused. If none available, then adp_tbl_oid is used.
72. Amdocs 6 (Clarify 13) + ADP Tables have been renamed and modified Lucky for us, new views have been created These views have the same names and structure as the old/original ADP tables So our queries still work the same
75. Postlude More Info on Dovetail SchemaEditor http://dovetal.es/dtSchema More Info (and free download) for Dovetail BOLT http://dovetal.es/dtBolt Feel free to contact me: gary@dovetailsoftware.com dovetailsoftware.com/blog/gsherman 512-610-5466 @gsherman These slides will be available on my blog at dovetailsoftware.com
I am the VP of productsWorking in Clarify for 15 years2 years as a customer, 4 years at Clarify as a Principal Consultant, 9 years here at DovetailI love sharing my knowledge of all things ClarifyMy blog at dovetailsoftware.comI actively participate in the Clarify forum on IT Toolbox
Create a view that contains case data, plus the number of open subcases for each case.http://www.dovetailsoftware.com/blogs/dbergondy/archive/2007/12/20/create-a-view-with-a-calculated-column-to-be-used-in-clarify-crm-and-dovetail-agentBaseline Clarify uses sql views with UNIONs quite a bit.For example, the baseline view wipelm_all (view #: 5748) is a SQL view in order to take advantage of the UNION syntax.Can “wrap” other linked tables, or tables from other databases, and query them via normal Clarify tools.
Notice:It has a name, and an id.It has a SQL statement that uses DISTINCTAnd it has a bunch of SQL View Columns
Notice:It has a name, and an id.It has a SQL statement that uses DISTINCTAnd it has a bunch of fields
fc_contact_view uses DISTINCT. This is a custom Dovetail view.Also have a look at the wipelm_all view. It unions a bunch of queries together. This is a Clarify baseline view.
The MTM relation is represented by an associated table between the two objects. This table consists of two columns -- one for each relation name. Clarify uses a unique naming convention for this table. The MTM table is a physical table in the database that normally has no objid of its own, so it is not listed in Data Dictionary for Objects. However, POMS and Dovetail SDK do know about this table.
This table acts like an MTM table in that it resolves the potential MTM relationship between Sites and Contacts -- sites may have many contacts, a contact may be related to more than one site.If it were only an MTM table, you wouldn’t be able to tell which site was the primary site for the contact.Clarify has many of these role objects. This is a common pattern.
In Rails, they call this Polymorphic AssociationsThese pseudo relations were a bit of a data modeling hack, before exclusive relations. Didn’t want to have to create a relation to every object that time_bomb could be related to.
These pseudo relations were a bit of a data modeling hack, before exclusive relations. Didn’t want to have to create a relation to every object that time_bomb could be related to.
Notice that r_rqst2case, r_rqst2bug, r_rqst2contact, etc. are all MTO relations.So we would expect to find them defined on the r_rqst table in the physical schema.Looking at the physical columns, we don’t see those defined.
The resource table has a change field.Whenever the resource state field is updated, the last_update timestamp will be automatically updated as well.
The case_history is marked as a CHANGE_FIELDThe modify_stmp is marked as a CHANGE_DATESo when the case history changes, the modify_stmp will be automatically updated.Of course, there’s a little flaw here – case_history stops updating after 32K, showing that having a text field as a change_field is probably not the best way to do this.
We covered this in schema 101
This ERD shows the 10 physical tables and their relationships.
indexes: these are application, or schema indexes, such as uniqueness indexes. Not performance indexes
I say “basic” algorithm, because there’s additional logic that occurs when you hit a rollover limit (i.e. 2^28)
If you’re on older versions of Clarify on Oracle, next_val comes from an Oracle sequence instead of adp_tbl_oid.
next_val is the next unique value for the given table. next_val for an object (type_id) is obtained first from adp_tbl_oid_unused. If none are available from that table then adp_tbl_oid is used.
