I woke up at 4:00am this morning in a cold sweat. And I haven’t slept since. In the wee hours of the morning my brain, as is its want, was mulling a worst-case scenario: “What if we got the image or imaging wrong. How are we ever going to re-image these 2575 machines?”

In our planning, the imaging process was always date constrained. Whilst David Haysom had planned an extra 2 days, just in case, as there are always strange things that could bite you on the bum.

First day of Netbook handout. Today. 9am. Within an hour, we had handed out about 10% of the total.

And a problem appeared. Thankfully not hardware; the opening and recharging of each machine tickled out any weird hardware issues. In fact, our out-of-box hardware failure rate was half of the planned amount. HP repaired these quickly, and all was well.

Boot to nothing. Ouch!

The problem: missing images on a low, single percentage number of machines.

After reviewing our WDS/MDT2010 imaging server and technique by local and Redmond experts: this was deemed not the root cause for the missing images. Another possibility was Netbooks booting in the box, running out of power, and getting into a weird state. Again, this was discounted as there was just no data on the drives.

The Current Theory

The imaging process ends with the Netbook shutting down. From F12 to network boot to shutdown is one keypress: F12 to “boot from network”. At the completion of the multi-cast and unpacking in the WDS/MDT process, the Netbook shuts itself down to a black screen.

On the first two days of imaging last week, the imaging area had power issues: essentially, fluctuating power at the setup trestle tables. This caused machines to be left powered off. To a black screen.

Our thinking is that the black screen at the end of this process led to false-positives. Netbooks that had power issues, left on a black screen, were accidentally deemed “imaged” during the first few days and packed up.

Learning 1:  Leave setup machines with a static, on-state saying “OK” would be our recommendation if you are planning such a large, time constrained install.

The QA process we had during the imaging week was to test every 80 to 100 machines: that is, go through the out of box experience to test build quality.

Learning 2: factor in a slightly higher rate of QA checks than you think necessary.

How are we Fixing this?

Handing over a known good, working Windows 7 Netbook to each attendee is a must. No compromise. Therefore, today, we instituted a quickly developed solution to freshly image these machines.

After re-evaluation this evening, we are dedicating a team to process any machines with missing images. There is also another, faster process that is being investigated that may be implemented (via USB imaging)

So, worst case: we have upset attendees whilst we fix their machines. All I can say is “Sorry, and we are making it right” and I am confident this process will work tomorrow. With a little patience, all will be well.

Best case: our current glass-half-full thinking is that the machines that were handed out today were from the early in the week batch: where we had power issues, and black screen false-positives.

Tomorrow will tell.

Thanks for your patience and understanding.

And if you want to complain or vent at someone: find me.

There will be a time, after TechEd, where we triage our original planning: time, budget, hand-offs, tools and techniques. As a Quality Manager at one stage in my working life: project review, documenting them and absorbing learnings are critical.

On a lighter note this Friday: here are two youtubes for your viewing pleasure:

After meeting Jack Morton’s Patrick, and donning a hi-vis vest – its into the Technical Learning Centre to witness the setup of the 2575 HP Mini-notes/netbooks.

There are about 13-14 long trestle tables, each with and average of 20 machines laid out. 8 staff are out and about unwrapping each box, putting it on the table, and initiating the Windows 7 TechEd image. This is a sight to behold. It feels somewhat like an Industrial Revolution-era factory. A study in time and motion.

Time is critical now. We cannot delay TechEd, nor stop the setup of the Technical Learning Centre – so its all hands to the wheel to complete the imaging tonight. Being a production process that is now in production, and costing per hour – it is way too late to change anything. Either the image, the process or technology involved in the imaging.

To give you a vision of what is going on – here is a quick youtube I created within 10 minutes of arrival.

Empty trestle tables, waiting for PCs in the Technical Learning Centre

Show machines setup

HP 2140’s being imaged and recharged

Jorke in his High-Vis Vest. How cute is he?

These photos were attached to an email from Simone Hulse:

Well it’s been an exciting few days for the build crew at Tech.Ed.  The technology team are very well established as they have been here for over a week.

• The mini-notes are imaging
• The show machines are complete
• CommNet and recharge is set in the foyers ready for APC (editor: Australian partner conference)
• The TLC is well underway (editor: TLC = Technical learning centre)
• Windows 7 Ideas Exchange lighting truss is in – and will be incredibly impressive once the lighting is complete
• The showcase banners and lighting rig is in
• The APC dinner staging is almost complete
• The rig is completely set in the roof, so now we can start building the expo from the ground up.
• Today is all about data cabling and power, tomorrow is about flooring and more cabling and then we build the expo on Friday

For those of you who are not hear yet.  The weather is warm in the day 22-30 degrees, but gets cool at night…. bring layers (and closed shoes!)

Very exciting.  Another edition later in the week.

Pop over and have your say.

At this stage we should note we leaving the Netbooks open for you to install whatever you like; Microsoft or non-Microsoft. These are your machines – and we’d love you to give Windows 7 RTM a really good test.

All of this said, the following LOLCAT appeared to me today. Don’t panic. There can be no delays.

see more Lolcats and funny pictures