Dispatches From The Geeks

News and Announcements from the MCS Systems Group

Update: CIS Maintenance Weekend this weekend.

Here’s the official list of affected services:

http://today.anl.gov/2016/01/it-maintenance-weekend-jan-15-17/

Note that Single Sign On (SSO) which allows external apps to authenticate against Argonne accounts (like Box, Workday, etc.) is being temporarily migrated offsite so that it will remain up during the network outage. And CIS is confident this outage won’t last more than 15 minutes.

Thanks!

Written by Craig Stacey

January 15, 2016 at 2:53 pm

Posted in Uncategorized

CIS Maintenance Weekend this weekend.

This is a reminder of the outage happening this weekend. The takeaway is still the same, anything behind the lab’s firewall will be unavailable for a brief time tomorrow morning, with some periods of up and down time throughout the morning. Things should all be normal by noon.

The official story from CIS is that e-mail and other services are not affected, but that’s not going to be the case. As such, the official announcement from them isn’t accurate and I don’t want to send it on since it implies a false sense of what the situation really is.

So be prepared for brief outages of lab-hosted services tomorrow between 8 and noon.

Thanks. My original notice follows below.

===

CIS Maintenance weekend is the weekend after this coming one. I normally forward these announcements on to you, but I noticed some possible errors in the list of affected services, and I want to get some clarification from them on it. I’ll send a complete list when I have it, however based on the fact that the Argonne network will be undergoing a major upgrade on the morning of Saturday, January 16, we can expect access to services from offsite that are behind the lab’s firewall to be inaccessible.

While this does not include services Systems runs here in building 240, nor does it include ALCF’s machines, it does include services hosted in building 221. This includes lab e-mail, lab web servers (includinghttp://www.mcs.anl.gov), and services that Systems runs in that building. Because that building has the most stable power situation (generator-backed UPS), we’ve housed all our critical servers there including Systems-provided websites and services such as WordPress, Mediawiki, Confluence, Jira, gitlab, etc.

The official outage window is from 8AM through noon, though the work is expected to take no more than an hour if things go as expected; the extra time is for the unexpected.

I’ll send an official list of affected services once it’s nailed down, but I wanted to get this word out quickly in case there are issues we need to be aware of regarding sustaining the outage at that time. If it poses a particular problem for you, let me know.

Written by Craig Stacey

January 15, 2016 at 8:30 am

Posted in Uncategorized

CIS Maintenance Weekend January 15-17

CIS Maintenance weekend is the weekend after this coming one. I normally forward these announcements on to you, but I noticed some possible errors in the list of affected services, and I want to get some clarification from them on it. I’ll send a complete list when I have it, however based on the fact that the Argonne network will be undergoing a major upgrade on the morning of Saturday, January 16, we can expect access to services from offsite that are behind the lab’s firewall to be inaccessible.

While this does not include services Systems runs here in building 240, nor does it include ALCF’s machines, it does include services hosted in building 221. This includes lab e-mail, lab web servers (including http://www.mcs.anl.gov), and services that Systems runs in that building. Because that building has the most stable power situation (generator-backed UPS), we’ve housed all our critical servers there including Systems-provided websites and services such as WordPress, Mediawiki, Confluence, Jira, gitlab, etc.

The official outage window is from 8AM through noon, though the work is expected to take no more than an hour if things go as expected; the extra time is for the unexpected.

I’ll send an official list of affected services once it’s nailed down, but I wanted to get this word out quickly in case there are issues we need to be aware of regarding sustaining the outage at that time. If it poses a particular problem for you, let me know.

Thanks.

Written by Craig Stacey

January 7, 2016 at 4:51 pm

Posted in Uncategorized

Phishing mail

Some of you may be getting very obvious phishing notices saying your anl.gov mail account is over limit. ANL Cyber has been notified, and you can safely ignore and delete the message. Thanks, and happy new year!

Written by Craig Stacey

January 4, 2016 at 9:53 am

Posted in Uncategorized

MCS Website fixed

In working with the ANL web team, we were finally able to locate the issue and correct it yesterday afternoon by around 5:20. A change made globally to http://www.anl.gov (which is where the main MCS website is hosted) to bring mobile support to the website and make some font changes had unexpected behavior on how our website was rendered. We’re still not sure exactly why the issue presented the way it did, but it was some interaction between the main website and our proxy running at http://www.mcs.anl.gov that caused an issue in rendering. The problem was not present on the testing website, which is why they didn’t believe there was an issue pushing out the update.

We’ll be meeting with the team in January to make sure further rollouts don’t hit us this way, as well as go over the plan for updating our proxy configuration.

Written by Craig Stacey

December 16, 2015 at 8:36 am

Posted in Uncategorized

MCS Website not displaying properly

Changes within the ANL webserver content seems to have broken the MCS website (hosted on their server). We’ve alerted CIS to the issue and are awaiting word on what happened and how to bring things back quickly.

Written by Craig Stacey

December 15, 2015 at 10:52 am

Posted in Uncategorized

FYI: Users may get an SSH host key warning when logging into login.mcs.anl.gov. This is expected as the key has been recently updated.

Written by Craig Stacey

December 9, 2015 at 1:15 pm

Posted in Uncategorized

File server outage

This morning’s power work unexpectedly took out one of our file servers (for reasons we have yet to understand). We’re in the process of bringing it back. This affects most home directories for Unix.

Written by Craig Stacey

December 7, 2015 at 9:58 am

Posted in Uncategorized

CIS Maintenance Weekend December 4th-6th, 2015

Please see the below announcement from CIS. There will be no impact on CELS systems.

WHAT ARE WE DOING?

Argonne’s quarterly IT maintenance weekend is scheduled for Friday, Dec 4th, thru Sunday, Dec 6th.

We are patching and upgrading our major application databases (Oracle, SQL, and MySQL) during the maintenance weekend. In addition, maintenance to systems not affected by the database work will also be performed.

Services Impacted: (> 4 hour maintenance length)

· All business systems and web applications will be unavailable throughout the duration of the maintenance weekend.

· Sharepoint site access (Dec 4th 5pm- Dec 5th 9am)

Services Impacted: (.5 to 4 hour maintenance length)

· Voicemail (Dec 4th 5pm- 7pm)

Services Impacted: ( <.5 hour maintenance)

· Paging

Services NOT Affected:

· Networking

· VPN

· Wireless

· Email Services

WHY ARE WE DOING IT?

CIS recognizes that the reliability and performance of Laboratory Information Technology resources is essential to our ability to carry out our scientific mission. Like the rest of our laboratory infrastructure, these systems require periodic maintenance, replacement and upgrades.

WHEN WILL THIS OCCUR?

December 4th, 2015, 5:00 p.m. thru December 6th, 2015, ~5:00 p.m.

· We expect Sharepoint Services to be restored by 9am on December 5th. An “all clear” will be sent to Sharepoint Site Owners after verification is complete.

· We expect verifications to be complete by Sunday at 5p.m. and an “all clear” message will then be sent.

Reporting issues (after maintenance weekend):

· Use the “Request Help” button thru the Argonne Service Desk interface https://help.anl.gov (tickets will be addressed by the Service Desk at the start of business (7:30am) on Monday, Dec 7th.

· Contact the Argonne Service Desk by phone at the start of business (7:30am) on Monday, Dec 7th.

Written by Craig Stacey

November 23, 2015 at 5:16 pm

Posted in Uncategorized

Systems Announce Some Mail Users Having Issues

This has just been received from CIS:

Email service should be restored to those whom were affected by the downed database. Search functionality will be restored hopefully by tomorrow morning once all the background processes have completed.

Should you have any further issues with your mail, please send mail to help@anl.gov.

From: <cels-systems-announce-bounces> on behalf of Tina Piersanti <tstanton>
Date: Monday, November 23, 2015 at 2:27 PM
To: CELS ANNOUNCE <cels-systems-announce>
Subject: [Systems Announce] Some Mail Users Having Issues

CIS has confirmed that they are having issues with a mail server. Only some Users seem to be affected by this issue.

More to come as we receive updates.

Written by Craig Stacey

November 23, 2015 at 3:18 pm

Posted in Uncategorized

Follow

Get every new post delivered to your Inbox.

Join 58 other followers