ICONUK Sep. 21&22, London – I’ll be speaking again


Just got this in the email:

Good News! Your session IBM Connections – Installing the Free “Extras” and  Integrating with other Products has been approved for ICON UK 2015
Tim and I are finalizing the schedule and it should be online tomorrow.

Thanks again for your submission and we are looking forward to seeing you  in September

Will I see you guys there?

Dates:
Monday 21st & Tuesday 22nd September 2015

Time:
Mon 08:00 – 19:00 (Gala dinner 20:00 – 23:00)
Tue 08:00 – 18:00

Location:
IBM Client Centre, IBM Southbank, Upper Ground, London

http://iconuk.org/

Lotus Notes 8.5.2 code error – The Tale of Regression


 

I am part of a Domino migration at a large client in the financial/insurance industry and we have been moving applications from Domino 6.5.5 to version 8.5.2 FP1. I know, there are newer versions out there but you can’t always upgrade blindly – this is going to be one of the stories why IN-DEPTH testing is so important.

The client has an application that has been around since R5 days, migrated and upgraded several times – both client and server versions and never an issue.

Suddenly we get reports (during testing – thank GOD!) that when using the app with the new client (LN 8.5.2 standard) that certain document do not get created – they vanish.

To make a L O N G story short – we opened a ticket with IBM support and after digging and prodding they produced this one for us:

(from the actual ticket)

******************* SUBSEQUENT CALL RECORD TEMPLATE  *******************
ACTION TAKEN:                                                           
Created a test DB in 85 and copied customer form and data.              
Found field “eml” caused the issue: SPR RDJS8APTK6.  

This is a regression error that made its way back into the Lotus notes 8.5.2 code stream. It was fixed in Notes 8.5.3 but if you are on 8.5.2 (no matter the FP level) you are snorked.

What we did

Since it is hard to determine with any certainty how many other applications out there might contain a filed called “eml” the decision was made to do a company wide, mid-project change and instead of rolling out Notes 8.5.2 we will be rolling out 8.5.3 and upgrading all workstations to which 8.5.2 has been deployed previously. A huge undertaking.

Had this not been found during testing it could have been very costly for my client.  As is, the change in scope will be massive and costly as well, but it beats having actual business processes interrupted and clients (actual consumers) be impacted.

So – the lessons everybody should take home from this:

  1. Test – always test even if it is a small point upgrade
  2. Test – with ALL server and client variations that you might have in your environment – do not skip anything
  3. Have test plans that are captured and documented across each scenario and can be compared apples-to-apples.
  4. DO NOT NAME ANY FIELDS, FORMS (or anything else for that matter)  IN YOUR APPLICATIONS “EML” – this error could possibly be re-introduced in a future code stream – you never know.

 

 

Lotus Notes 8.5.2 -> FP1 released and ready for download


I have been a bit preoccupied with a troublesome BES installation for the last week and have not been looking much at the web and blogs – but today it looks like FP1 for Lotus Notes 8.5.2 is out and available. I am already downloading it and will apply it to all my many clients today.

Can’t wait until the FP1 for the server is out.

Domino 8.5.1 Technote – Fixup can crash your server


I came across this technote yesterday and investigated it right away …

– well, I’ll admit it – I asked somebody else in IBM to clarify some items for me and am now plagiarizing that good deed on my blog – THANKS CAROL!!

First of all – this only applies to Domino 8.5.1 on Windows, and it makes no difference which FP you are running since it was not fixed in FP4 nor will a fix be included in FP5 either, it is fixed in 8.5.2 though – you guys are safe.

 

The Issue Explained:

The issue CAN be cause by running FIXUP with a combination of either [fixup -F -J] or [fixup -F -J -O]. It will not automatically happen, it has to run into a document in a dB that has ‘invalid items” .. what exactly constitues such “invalid items” is not 100% clear but I am treating it like a 50-50 chance for failure until I have more evidence/experience or a better explanation of what exactly is meant with “invalid items”.

Exception:

Since the explanation implies that the behavior only happens if you also include the [-J] switch, servers that are not not transaction logged will not be affected, even if they are on Windows. Also, if you are running Domino on any other platform this will not affect you – so my dear Linux buddies are all safe, as are you AIX, iSeries and mainframers as well.

 

What does this mean in terms of daily admin work?

Well, the [-J] is needed anytime you want to run fixup against transaction-logged databases and [-F] is a very commonly used switch as  well. [-O] basically tells fixup to also run against open databases and is the default bahavior if you specify a database name with the fixup command.So the command [load fixup mail/xxx.nsf -J -F] implicitly includes the [-O] switch behavior whether you add it to the command or not.

That leaves you with the option to run Fixup without the [-F] switch to avoid a  *possible* kiss of death.

 

Advice:

if you are running the fixup command as part of some regular, automated script (I usually advise against it) you need to review the commands to make sure yo do not inadvertently crash your server (possibly, it is not a guaranteed outcome).

Also, when running Fixup against databases during the day to fix database corruption of any kind, I would run it without the [-F] switch during the day and see if your problem is fixed. If that does not fix your database problem, you can either roll the dice or wait until after hours – which is what I will do until I have tried it out a few times and have a better feel whether this actually ever materializes or not.

Ridding myself of Windows – Google Desktop for Lotus Notes and Linux


Again, research, research – I came across this technote today:

IBM – Selecting Actions – Google Desktop for Lotus Notes yields error for Google Desktop plugin on Windows 7.

That reminded me of my current struggle of eliminating Windows from my laptop. Quick reminder – I don’t hate Windows, not do I knock is in personal or professional conversations at all – I just prefer Linux (currently Ubuntu 10.04, will upgrade to 10.10 S O O N) and I prefer not to pay allot of money for a first rate OS and all the software I want to use. I have another Windows Desktop at home, several servers and run Windows as a VM on my Linux machine when I need it.

Google Desktop:

First off let me say that I do applaud IBM for adding technotes about a product like Google Desktop that they don’t produce themselves – that is a great thing that I I hope they keep up.

I started using Google Desktop a few years ago and fell in love with it instantly. The native Windows search-your-desktop product regularly crashed my machine, and after I switched – I never looked back. I also Love the fact that IN WINDOWS it has a plug-in so that you can include your Lotus Notes based email in any search you are conducting. Brilliant! However, the Linux Google Desktop does not support this … sadly.

As to the why? Well, many reasons probably but the major one being the low number of Linux desktops that are used by real end-users out there and not just IT professionals.  For Google to put in the effort they are probably hoping for IBM to put more effort into that field.

David Gewirtz wrote a very interesting article in the Dominopower online magazine recently regarding this very topic. He made a very good point that the latest version of Lotus notes 8.5.2 is only certified to run on three distros: Red Hat, SUSE and Ubuntu. When it comes to Ubuntu – the most vibrant distro for desktops – the supported version is literally 2 years old. I have it running on the latest version (10.04) but I would never implement this in a customer environment as the desktop configuration would be unsupported by IBM.

Clearly, there is some room for more effort on IBM’s behalf to put some more resources into the Linux portion. I will even stop my griping about the lack of native Admin and Designer clients on Linux (note: my work-around tip is here)  and happily skip-and-jump down the yellow brick road for them. I will have to admit though there is a bit of a chicken and egg situation here: because there is not allot of pull by the market and free-bee extras (like Google Desktop Notes Mail integration) out there IBM is not pushing Linux nearly as hard as they could (or as I would wish them to – different matter I guess)  which in turn does not inspire other companies to put allot of effort into developing Linux variants of their products. Catch 22, viscous cycle, etc.

But clearly IBM is the one who would be able to make more waves in this field if they at least added more Linux distros to the supported list and – especially for Ubuntu – made sure they were not 2 versions behind. Or – novel idea – put out a time-line, schedule (or whatever) that shows where they intend to go with supported Linux versions, distros etc. AND they would have to put it someplace where it can actually be found … I searched but found nada. Just a few posts by TEB alone don’t make a compelling Linux argument. If you don’t have your information someplace where a Google search (or any other search provider) does not show it on page one or two of the results – it does not count. (note: I was searching for a good 30 minutes, that is longer than the average customer would be willing to spend time on a search)

** End of Linux-Griping for the day **

Hidden Gems in Domino 8.5.2: Overview of Improvements in DAOS Catalog


Another great find (this tech note just came out today) of some improvements that have found their way into DAOS in 8.5.2.

I am really liking DAOS now (I always have, it’s more an increase of  like-iness) . It is so much easier to sell a client on DAOS now as the feature has been around for a while and you can show and document the evolution of issues, their resolution and the current state of the product. Along with the DAOS estimator that kind of documentation is a powerful tool that can be used to overcome allot of rather irrational fears.

IBM – Overview of 8.5.2 Improvements in DAOS Catalog.

Many clients are – understandable – hesitant to do anything with attachments … it gets them all nervous. Once you can show them the advantages though it becomes easier – especially if you start talking of performance increases on the server and that even if they get nervous – DAOS is bound to each server individually and that makes them feel safer if some other server has the “full” (a misnomer, but effective) replica.

I like to introduce it on to servers that are used for archiving, back-up and redundancy. Back-up times are quicker, tapes are fewer, AV scans are MUCH quicker and the server (usually not the fasted one in the crowd) performs better. A great way to get a client comfortable and then have them implement it system wide.

DCLug Meeting – 09/23/2010


I have been working for a client full time in DC for a few months now and some members of the DC Lug are at the same place – so after some beating and whipping by them I joined the DC Lotus User Group a month ago or do (it still hurts …)

Now, for the first time in a while (should I take credit??) there will be a DCLug meeting on Sept. 23 3-5. The meeting details are here.

There is also a DC Lotus Professionals group on LinkedIn that I think is a great resource to join as well. If you have a LinkedIn profile (I hope you do) and are anywhere in the greater Washington D.C. area, you should join that group.  Here is a link to join that group.

LotusLive – some questions not asked …


LotusLive – IBM’s entry into the whole cloud computing and SAAS field. Arguably Domino has had this technical capability to work in “a” or “the” cloud for years though there were no buzz words attached to it and – IMHO – the market would not have been ready for the whole thought process. Though there are a few hosting companies (IBM among them, or Connectria, Promenic, …) that have made a good living servicing this need before Google and Microsoft came along and claimed the patent on it.

I assume that most readers of this blog will also – at least occasionally – listen to Lotus technology related podcasts. Two of my favorite Lotus related podcasts are This Week in Lotus and the Taking Notes podcasts. Both are well by their organizers, have dubious side-kicks(*) that assist the grand pod-masters and cover some really interesting technology and trends in great detail.

I listen to podcasts allot on my way to work and back in the subway or, if they are longer (both of these go about an hour) in the car on the weekend driving back-and-forth between home and DC. so this weekend I got a double whammo of listening to TEB (The Ed Brill) twice in the span of two days as he was on both of these casts going into more detail of the latest LotusLive announcements, especially LotusLive Notes.

LotusLive:

Great feature set, good marketing, excellent price … wait, price – costs – there is something missing here. Timing of the release of these two podcasts made me first listen to Bruce and Julian (Taking Notes) talk to TEB and after it was done something was nagging me .. there were some questions missing that I had in the back of my mind ever since I heard the “official” announcements TEB had for us at IamLUG. Some of the blog entries Chris Miller wrote about Connectria already being in this hosting/SAAS/Cloud computing space for years were also racing around in my mind but it was close to midnight and I was trying not to get killed on the Jersey Turnpike so I just put those thoughts aside.

The Unanswered questions:

Sunday night I drive back south to DC and listen to Stuart (main man) and Darren (side-kick) 😉 talk to TEB and this time it came to me. Or maybe waiting in traffic after crossing the Delaware bridge to go through the toll booths (why does not EVERYBODY have an EZPass??????? make it mandatory!!!) jogged somethin in my mind – in any case, the thing that was nagging now became clear: everybody was touting the $5.00/user/month rate for clients and the minimum of 25 users per company for LotusLive Notes. Great cost, easy to deal with … yeah, until the words “integration”, “competitive offers” and “Client provisioning” came up and the fact that TEB mentioned that bringing a client on takes some provisioning work on the back-end.

Well, integration – that is what I do for a living and have done for many years. The process of bringing an existing Domino/Notes infrastructure into LotusLive Notes – along with all the history and garbage that probably exists: older version clients, inconsistent client configuration, specialized mail routing, existing internet mail access for clients, VPNs, users that are VEHEMENTLY against local replicas (I love local replicas, unfortunately allot of clients don’t understand why this is considered “BEST PRACTICE”) and lack of experienced IT staff – or just a lack of enough qualified IT staff, etc., etc., etc. …..

Integrating an existing environment takes time and effort – time and effort directly translate into $money$. Everybody was always talking about how Google sold their product with the slogan “$50/user will take all your probelms away” until clients had to go through the actual process to get all of their users into the cloud … just look at the city of Los Angeles. Other questions such as how long such a move can take (on average, every client is different) and what kind of architectural changes (more details, I only heard something about OUs) are going to be necessary would have been great. But then again – I am a technical guy who very well might find myself on a project moving a customer to LotusLive Notes someday and am naturally interested in these details.

What nobody was asking was what the average project to bring clients “into the LotusLive Cloud” was REALLY costing – either as an average per user or per project. Nor was anybody asking about what kind of contract clients will need to sign in terms of length of contract. Just like phone companies give you that phone at a reduced price but require you to sign a minimum 2 yr contract to recoup the expense I assume that IBM needs to look at something similar – or can you just take your 25 people and leave after 4 months if it just ain’t what you had expected?

The reason I am asking this is that this creates a per user cost that has to be taken into consideration by the client … when does the break-even point arrive? How are my licenses that I had up until now going to impact my costs? TEB eluded briefly to the fact that IBM will take a clients contribution to licensing into consideration … but what if that move to the cloud now takes license sales away from business partners? Is there something in the deal for them or does IBM just poach clients and tell the business partners to go and look for some new business … I assume not but then again this topic was not covered.

Another question I had was whether IBM will just take those integration services, hand them to ISSL and that be it or will a business partner that sells LotusLive seats OR the business partner that was servicing this client for years and did all their licensing also have the opportunity to do the integration work and benefit from the existing expertise and experience of IBM in this field and maybe make this kind of work into a major part of their business? If IBM takes business on one side, they will have to give something else in return or they are likely to loose partners down the road.

In all fairness – Stuart did put something out on his Facebook page and asked for people to contribute questions for the podcast and I did not .. I only wrote a half-witty answer to something my favorite little Scottsman of HADSL fame wrote so I can’t blame anyone but myself on this one. Also, the podcast is only one hour long and you can’ ask EVERYTHING in one hour, even if you talk as fast as I do – and most people don’t.

So, why am I nagging?

So, why am I writing this? Well, there are some more pod casts put there and TEB still has breath left to talk (I think) so somebody please get him on their show and ask some more technical and monetary questions. Or, I also assume there are some white papers, technical documentation that might already be out there which covers this but nobody has been able to easily find on IBM’s website (a common issue, but then again I did not put allot of effort into searching for them prior to writing this either).

Or, we can get TEB to fire up his iPad and add a few posts to his blog where he either goes into this detail or points into the direction of information where the answers to these questions and more could be found?

Details? Thoughts? Dementis? Opinions? – Anyone got something for me?

(*) My stab at the “Dubious Side-Kicks” was totally UNCALLED for – they do a better job than I would at their tiny, insignificant roles – but I just could not resist .. I beg forgiveness gentlemen!

Disclaimer:

I am writing this at 1:30 AM and no animals were harmed in the process of mangling the English language,  nor am I on the take of Google though I do like their Reader software and use Google Voice. I also must mention that I like cupcakes – though not at 1:30 AM

Domino: DAOS & Transactional logging error – “…Log file is full” results in server hang


While perusing the IBM site and reviewing new technotes and SPRs I came across this one: 1440744

Basically it pertains to DAOS and transactional logging and a potential issue. The work-around is classic – call IBM and see if they can create a HF for you … wonderful. The reported SW version is 8.5 and the wording is

“…is expected to be fixed in an upcoming release of Lotus Notes/Domino”

so now I’ll go through the fix lists of 8.5.1 and (hopefully soon) 8.5.2 to see if this SPR was fixed there.

******************* Technote details ************************

Error: “…Log file is full” results in server hang

Technote (troubleshooting)

Problem

In Lotus Domino, transaction logging and Domino Attachment and Object Service (DAOS) are enabled. However, every day the server hangs with the following error message; “Recovery Manager: Log File is Full”, necessitating a restart of the server.

Sample error in the console log:

[373070:00094-23131] RmFlush: Error calling DbDirGetBlockByDbiid for

DBIID E384E10C [span = 1649084K recLSN = 00002605-A2C3AE74] : Entry not found in index

[373070:00094-23131] RmFlush: Error calling DbDirGetBlockByDbiid for

DBIID E384E10C [span = 1690445K recLSN = 00002605-A2C3AE74] : Entry not found in index

……

[373070:00094-23131] RmFlush: Error calling DbDirGetBlockByDbiid for

DBIID E384E10C [span = 4000889K recLSN = 00002605-A2C3AE74] : Entry not found in index

When the span reaches 4G, the following error appears:

[495858:00002-00001] [datetime] Unable to update entry in catalog (for mail/xxx.nsf): Recovery Manager: Log file is full

Resolving the problem

This issue was reported to Quality Engineering as SPR# FLII8453UR and is expected to be fixed in an upcoming release of Lotus Notes/Domino. If you experience this situation, contact Product Support to see if a hotfix is available for your particular configuration.

via IBM – Error: “…Log file is full” results in server hang.

IamLUG: What comes next? Consultant in your Pocket


This is a place for a shameless plug: IamLUG is free, you most likely did not come because you are so cheap, free is still to expensive as it includes travel expenses. Or, maybe – if you are lucky, you can deflect blame to your company that they are too cheap. In any case, you are missing out on good education and a chance to learn something new and drink allot in the evening.

So, now you are in the hunt fo the next best thing: free webinars …. enter the “Consultant in your pocket” series.You can sit in front of your computer, wear your fluffy bunny slippers and do not need to shave or brush your teeth – it does not get any better than that.

Here the shameless lug: I will be hosting one of those on Sept. 1st about Lotus Protector. Almost as good as IamLUG – maybe better as you will not have to look at my but only will need to listen.