Dispatches From The Geeks

News and Announcements from the MCS Systems Group

Mail Slowness

An over-aggressive DOE scanner spam-bombed us this afternoon, bringing our mail service to a crawl as three mail servers tried to deal with over 40,000 e-mail messages sent within a matter of minutes.We believe we got it all stopped, and all the backlog of mail should be delivered at this point.Sorry for the inconvenience.

Written by Craig Stacey

October 9, 2009 at 6:35 am

Posted in Uncategorized

Tagged with

Pictures from the Datacenter move

Pics taken by Max.All in all, it took a little less than 11 hours. Everything went generally smoothly. It bodes very well for the move in two weeks!

Written by Craig Stacey

October 6, 2009 at 1:13 am

Posted in Uncategorized

Tagged with

First Datacenter Move

The first phase of the move of equipment from our old datacenter (the BMR) to the new datacenter (the Core) is complete, from a physical perspective. All the racks made it over with no issues.On Friday, our smallest cluster was up and running. Today, as the day wore on, we got more equipment up. As a sort of “dry run” for the big move in two weeks, this bodes very well. We’ll keep plugging away at getting the random pieces up and running, but there are no major stumbling blocks.Hooray!We’ll post a wrap-up on or before Monday.

Written by Craig Stacey

October 4, 2009 at 3:37 am

Posted in Uncategorized

History

Today, the first computer came up in our new datacenter, The Core. The cosmea cluster came up, largely without incident, and backed itself up to the tape library (that is, incidentally, still sitting in 221).Tomorrow’s gonna be a fun day.

Written by Craig Stacey

October 3, 2009 at 6:19 am

Posted in Uncategorized

Datacenter Move, Episode I, The Phantom Menace

Okay, folks, this is it. A little over two weeks ago, we moved a bunch of people into the spiffy new building. And this Friday, we begin the process of moving our computers over from the BMR to the Core (which is now what we’re calling the SSF).Are we nervous?Yes. Yes, we are. How could we not be? You know the old curse; May you live in interesting times. Well, this is a pretty interesting time.But we’re also confident. All of our plans, our timelines, our probable outcomes are unraveling, intertwining, recombining, and falling generally into place. By Saturday evening, we should have a pretty solid idea of exactly how well we did on the whole ordeal, and how the second and larger part of the move will go.As a reminder, the sysadmin of the particular system will send out a more detailed notice, but here’s the big picture of what’s moving:

  • Remaining TeraGrid infrastructure
  • NMPDR and IGSB infrastructure
  • The Cosmea compute cluster
  • LCRC PVFS fileservers

We’ll also be moving the bulk of our storage and parts. But that’s just our pain, not yours.In each of the above cases, the resource in question will go down on the Friday before the move (at some point in the day, depending on the resource), and get prepped for move. First thing Saturday morning, the movers will move the equipment into the Core, at which point we’ll start hooking things back up and have things operational as soon as possible.This is going to be an experience!

Written by Craig Stacey

September 28, 2009 at 11:14 pm

Posted in Uncategorized

Tagged with

Upcoming moves and Updates

Things have generally been “steady-state” with regards to the office moves, and we’re picking up the last pieces. Some notes.

  • The copier/printer on 4 (4171) is still awaiting its PostScript module. (I referenced the wrong printer in my last post.)
  • A color copier/printer will be installed at 2D11, but is not here yet.
  • Fax modules will get installed such that there’s at least one copier that can send faxes per floor.
  • The big color plotter is awaiting repair. Remember to check with the help desk if you want to print to it.

We’ve had one full week in the new building, and now our group’s attention must turn back to 221. In slightly less than two weeks, we begin the process of moving our computing infrastructure from the datacenters in 221 (known affectionately as the BMR and the LMR) into the ginormous datacenter here in 240 (known affectionately to me as The Computorium).As such, a lot of our concentration and effort will be going towards getting things packed up or retired. Then, on Friday, October 2, the first move will happen. Two weeks later, the second and last move will happen. We’ve got our schedule of moves set up, so here’s what to expect in terms of moves and downtimes:Move I: October 2-4

  • Remaining TeraGrid infrastructure
  • NMPDR and IGSB infrastructure
  • The Cosmea compute cluster
  • LCRC PVFS fileservers

Move II: October 16-18

  • Full Disclosure
  • Breadboard
  • kbT compute cluster
  • I2U2 resources
  • LCRC DDN storage system
  • MCS Core Computing infrastructure

In each of these cases, the resource in question will go down on the Friday before the move (at some point in the day, depending on the resource), and get prepped for move. First thing Saturday morning, the movers will move the equipment into the Computorium, at which point we’ll start hooking things back up and have things operational as soon as possible.I’ll leave it to the administrators of the individual resources to prep you for what you can expect, but as indicated above, you should generally not expect the resource to be available until the Monday following the move (though we’ll obviously strive for getting things up as quickly as possible).Of note is the second move, which contains the MCS core computing resources. Here’s a general schedule of what you can expect for that on the weekend of October 16-18.

  • October 16
    • 5:00 PM: general core computing resources go down. At this point, all login nodes, compute machines, file servers, etc., will be offline as we pull cables and prepare the machines for the move.
    • Note: This should not affect mail service. Reading and sending mail will definitely be unaffected, receiving of new mail may be delayed, but we’re working to avoid that.
  • October 17
    • Very early in the morning, movers start moving gear to the Computorium.
    • If all goes well with no snags, we should be operational before 5:00 PM, however the outage window is still until Monday in case things go other than smoothly.

There you have it! We’re shooting to keep downtimes to a minimum and make these weekends go as smoothly for you (and *us*) as possible. Sorry for any inconvenience this may cause you, and if you see a showstopper in one of the dates above, let us know ASAP.I’m looking forward to seeing computers in that big ole room across the atrium!Here’s to Fortune’s favor, may we find it!

Written by Craig Stacey

September 21, 2009 at 11:18 pm

Posted in Uncategorized

Tagged with ,

Moving Madness

Three full days in the new digs. We’re pretty much settled in, things are becoming normal, and we’re getting comfortable in our new surroundings. Thus far, there have been no zombie attacks from the rock gardens, so things are definitely starting out on a positive note.Some notes:

  • We’ve got a new wiki page up for MCS and ALCF users here. That’s where you’ll find any known issues.
  • We’ve added a new blog category, Known Issues, where we’ll announce updates on known issues.

In the spirit of reporting known issues, all the printers except for pr-4165 should be operational. We’re waiting on a PostScript module for the 4165 copier in order to make it useful as a printer. Otherwise, you should be good to go with any of your printing or scanning needs. Fax should be generally available soon on each floor, and we’re going to have a desktop package soon available so you can fax directly from your Mac or Windows computer. More updates on that as they arrive.The conference rooms have been updated in Zimbra to reflect accurate capacity counts, actual room number, building, and floor information. As soon as furniture arrives, we’ll be set!I intend to post updates on the order of every other day for the next little while. Also, I’m hoping you’ll see entries from more than just me. As always, continue to report your IT issues to issues to systems@mcs.anl.gov, and we’ll get to it.Thanks!

Written by Craig Stacey

September 17, 2009 at 9:47 am

Posted in Uncategorized

Tagged with ,

Printing and Wireless

The wireless network in TCS is slightly different from the one in 221, and we have to open up some firewall holes to allow printing from guest wireless. This should be working soon, in the short term use Auth wireless (which you really should use anyway, since it gets you behind the firewall), the VPN, or plug in.

Written by Craig Stacey

September 15, 2009 at 4:08 am

Posted in Uncategorized

Tagged with

Mailing List Server out of disk space

Our mailing list server ran out of space on 9/12/09. We’ve made some quick fixes to free up some, and will have the problem addressed permanently on Monday.

Written by Craig Stacey

September 14, 2009 at 2:54 am

Posted in Uncategorized

Tagged with

TCS Tour Take Two

Simple walkthrough of the 2nd floor of TCS.TCS Tour Take Two.

Written by Craig Stacey

August 21, 2009 at 1:16 am

Posted in Uncategorized

Follow

Get every new post delivered to your Inbox.

Join 40 other followers