« To all the vegetarians at Lotusphere... | Main| Perfect day in the life of a Developer vs an Admin »

A better way to simulate @Today/@Now in a view without killing performance

Tags: Domino Performance

In Kevin Marshall's and my session  BP208 Go Domino Go! Application Performance Engineering for IBM Lotus Domino Developers, I discussed why auto-indexed views with @Today or @Now in column or selection formulas take up lots of reindexing time.  You usually don't notice the performance hit during development and testing with only 100 or so documents, but it absolutely kills performance of the entire server once such a view grows to thousands of documents.
A picture named M2

Developers are tempted to use @Today (or @Now) in view formulas when they, for example, have to create a view that shows all documents created in the last 5 days...
SELECT @Adjust(CreatedDate; 0; 0; 5; 0; 0; 0) > @Today

One way around this I proposed in the session is to use a daily agent to move pertinent document into or out of a folder, which replaces the view.  That works, but sometimes you really need the @Today / @Now for a column icon or more complex selection logic.  

Public speaking rule #8: someone in the audience knows more than you.
An attendee at our session came up to me afterward and offered this nice suggestion...  He said simply use the formula above, but hard-code today's date [01/30/2008] in place of @Today.  Additionally, write a scheduled agent to change the view formula programmatically each night, so that the hard-coded date is updated each night.  The view will reindex each night, but be fast throughout the next day's usage.

I had to share such a great idea.  Everytime I present a session to a keen audience, I learn something new myself and this was no different.  Wish I had gotten the name of who suggested it.  Please let me know if it was you.

Comments

Gravatar Image1 - Hi Jamie! Good to see you posting. Well, it wasn't me, but here's a good link / comment thread re views and the use of @Today:

{ Link }

Gravatar Image2 - Great session. I'm looking at page 17 of the PDF. On the subject of @Userroles you have "Use in a computed field, then reference from elsewhere".

Will that work correctly? Or will it give a person viewing the doc the same user roles as the last person who caused the document to be computed?

My own testing has indicated that I need to make the field "Computed for display" to make this work.

Thoughts?

Gravatar Image3 - Great to see you blogging. I look forward to reading more (and will add you to my blogroll which is currently somewhat sparse).

Gravatar Image4 - @2 - Hi Tim, thanks for attending the session. Yes, it would have to be a CFD field if you want it to compute when the document is opened. Note that @UserRoles is not necessarily recomputed at each occurence as of ND7. Therefore, also note that we actually had removed that slide from our presented version. We had to turn in the Lotusphere version a couple of months earlier and before we had tested as thoroughly on ND7 and ND8.

Gravatar Image5 - No wonder I didn't remember that slide from the session. I was wondering how I missed that.

"Note that @UserRoles is not necessarily recomputed at each occurence as of ND7."

Which means I shouldn't try to use this at all or ...?

Gravatar Image6 - @1 - Ben, that's a great thread, thanks for pointing that out. I like the @Text( ) method, which is also similar to just making the View refresh frequency set to "Manual" instead of the default "Automatic". I've seen that before, but wonder if there's a drawback when using it with @Now in the freshness of the view calculations. Looks like you'd have to manually trigger the view to refresh if the data was not automatically being updated regularly.

@5 - it's still good practice to write your logic in one place (CFD field) and reference it as often as needed. But my comment about ND7 is that consolidating such @UserRoles calls into once place is no longer a performance gain as far as I understand it. Lotus made Domino faster by caching @UserRoles called more than once on the same form.

Gravatar Image7 - @6 - Ah. Thanks for the clarification. Since we are on ND7.02, I'm not going to gain anything by rewriting my existing H/W formulas. I was about to do a lot of work for nothing.

On a separate note, but the same presentation, you mentioned how expensive db.GetView is if you use aliases. Well everybody I talk to was pretty stunned by this because we all use aliases due to the business owners always wanting to change view names. Obviously I can avoid this by never using public views for lookups, but that requires duplicating views and that cure appears worse that the disease in terms of overall server performance.

Any idea if there has every been talk of adding "db.GetViewByAlias" or a server setting to first search for the view by alias, then by view name? Or is there a better way?

Gravatar Image8 - I got the heads up from my mate Bill

Welcome to the Blogging world !

Coatsie

Gravatar Image9 - Mick Baitinger whispered this to me during your session but maybe someone else had the same idea.

Gravatar Image10 - @7 - Tim, the effect is most noticeable from Notes (not as much from web), hundreds of views, when you are using a case-mismatched alias, and when the server is overloaded. Case matched alias is one fewer pass at the view list, but still two more than getting by the view name. And of course, you don't want to include getView in a loop where it could be set once instead. I suggest profiling your code to see your actual getview timing. It's one of those tradeoff situations between maintainability and performance. If you don't see a problem, then I'd stick with the alias.

Gravatar Image11 - @10 - Thanks.
I don't have hundreds of views in one db (you do mean in just one db right?) and I never write case-mismatched anything (too anal-retentive), so maybe it doesn't matter, but only profiling will tell.

But it does appear that Lotus is penalizing those who are using what would otherwise be best practice.

Since I always want to have my cake and eat it too, I'm tempted to propose a optional parameter for getview: db.GetView(string, [aliasfirst])

Gravatar Image12 - Andre Guirard has more good points on this topic here:
{ Link }

Gravatar Image13 - One problem/issue. I guess if you rely on a view that has it's formula programatically changed by an overnight agent, if you ever replaced the design of that database during the day from a template, you'd have to make sure that the agent runs at the time the design is replaced. Otherwise the view will be incorrect.

Gravatar Image14 - @13 - Hi Richard, that's true. One way you could do this is to have a database postopen event check one such known view, then set a flag in a db profile doc (to prevent that process from checking again next time). Not pretty, but it would automate the rollout step you identify. See some of Andre's points above (12) for other ways to do this, as well.

Gravatar Image15 - I think the way it works is if the agent is scheduled and enabled in the template, when you refresh the design the agent will run in the target database. Most people disable agents in templates because of this (as well as not wanting agents to run in templates), but you could use it to your benefit here.

Gravatar Image16 - an option might be having an agent that moves documents to s specific folder in stead of using a field with a flag ?Emoticon

Gravatar Image17 - Hi Jamie, welcome to the Blogosphere! Funny thing about this place is that rule #1 here is similar to that for public speaking: "Some else in the Blogosphere has probably already blogged your topic."

In this case, Sean Burgess posted code { Link } last year that does exactly what you describe here using a hard-coded date and scheduled agent. This is something I picked up on { Link }
with the goal of making an "improvement" to the approach that was designed to handle different international date formats, only to be informed by Andre Guirard that it was unnecessary.

I liked this approach so much I incorporated it into SuperNTF. I'll probably now have to implement Charles suggestion to enable that agent in the template.

All goes to show great minds DO think alike Emoticon

Gravatar Image18 - @16- Patrick, Not sure what you're asking/telling, but one way is to use an agent to set a field to cause documents to display their time value (e.g., "5 days overdue". However, this causes additional replication. Another way is to use a folder to move docs into or out of a folder, which is then used in place of the view when you need to see time critical documents. this has virtually no replication effects because documents are not modified when they are moved between folders. The other way mentioned here in this post works well in some cases. In most cases, simply changing the Index refresh frequency (e.g., to "Auto- at most every 24 hours") in the View design element is sufficient. See Andre's post in #1 above.

@17 Pettitt - you are so right. I'm still catching up on good stuff from a year ago. thanks for the comments.

Post A Comment

:-D:-o:-p:-x:-(:-):-\:angry::cool::cry::emb::grin::huh::laugh::lips::rolleyes:;-)