I found this handy how-to tutorial guide for "Talking to the Milnet NOC" and resetting the LH/DH, which was useful for guiding the NSA employee on the other end of the phone through fixing their end of the problem. What it doesn't mention is that the key box with the chase key was extremely easy to pick with a paperclip.
Who would answer the Milnet NOC's 24-hour phone was hit or miss: Some were more helpful and knowledgeable than others, others were quite uptight.
Once I told the guy who answered, "Hi, this is the University of Maryland. Our connection to the NSA IMP seems to be down." He barked back: "You can't say that on the telephone! Are you calling on a blue phone?" (I can't remember the exact color, except that it wasn't red: that I would have remembered). I said, "You can't say NSA??! This is a green phone, but there's a black phone in the other room that I could call you back on, but then I couldn't see the hardware." And he said "No, I mean a voice secure line!" I replied, "You do know that this is a university, don't you? We only have black and green phones."
Date: Thu, 11 Sep 86 13:53:45 EDT
From: Steve D. Miller <steve@brillig.umd.edu>
To: staff@mimsy.umd.edu
Subject: Talking to the Milnet NOC
This message is intended to be a brief tutorial/compendium of
information you probably want to know if you need to see about
getting the LH/DH thingy (and us) talking to the world.
First, you need the following numbers:
(1) Our IMP number (57),
(2) Mimsy's milnet host address (26.2.0.57),
(3) The circuit number for our link to the NSA
(DSEP07500-057)
(4) The NOC number itself (692-5726).
Second, you need to know something about the hardware. There
are three pieces of hardware that make up our side of the link:
the LH/DH itself, the ECU, and the modem. The LH/DH and the
ECU are the things in the vax lab by brillig; the ECU is the
thing on top (with the switches), and the LH/DH is the thing
on the bottom. The normal state is to have the four red LEDs
on the ECU on and the Host Master Ready, HRY, Imp Master Ready,
and IRY lights on at the LH/DH. If these lights are not on,
something is wrong. If mimsy is down, then we'll only have some
of the lights on, but that should fix itself when mimsy comes up.
Some interesting buttons or switches on the ECU are:
reset - resets something or another
stop - stops something or another
start - restarts something or another
local loopback -- two switches and two leds; you may need
to throw one or the other of these if the NOC asks
you to. These loopback switches should be distinguished
from those on the modem itself.
remote loopback -- like local loopback, but does something else.
The modem is in the phone room beside the terminal room (rm.
4322, if memory serves). It can be opened with the chase key from
the key box...but if someone official and outside of staff asks
you that, you probably shouldn't admit to it. It has a switch on
it, too; it seems that switch normally rests in the middle, and
there's a "LL" setting to the left which I assume puts the modem in
local loopback mode.
Now that you have some idea of where things are, call the NOC.
Identify yourself as from the University of Maryland, and say that
we're not talking to the outside world. They will probably ask for
our Milnet address or the number of the IMP we're connected to,
and will then poke about and see what's happening. They will ask
you to do various things; ask if you're not sure what they mean,
but the background info above should help in puzzling it out.
Hopefully, this will make it easier to find people to fix
our net problems in the future; it's still hard to do 'cause
we have so little info (no hardware manual, for example),
but this should give us a fighting chance.
-Steve
To: fair@ucbarpa.berkeley.edu (Erik E. Fair)
Cc: ucdavis!ccohesh@ucbvax.berkeley.edu, Hackers_Guild@ucbvax.berkeley.edu
Subject: Re: a question of definition
Date: Thu, 29 Jan 87 12:29:36 PST
From: Milo S. Medin (NASA ARC Code ED) <medin@orion.arpa>
Actually its:
SCINET -- Secret Compartmented Information Net (if you don't know what
compartmented means, you don't need to ask)
DODIIS -- DoD Intelligence Information Net
The other stuff I think is right, at least without me looking things
up. I probably shouldn't have brought this subject of the secure part
of the DDN up. People like being low key about such things...
Erik, all the BBN gateways on MILNET and ARPANET currently comprise
the core, not just mailbridges. Some are used as site gateways, others
as EGP neighbors, etc... And just because you are dual homed doesn't mean
you get a mailbridge. And the IETF doesn't deal with low level stuff
like that; DCA does all that. In fact, the reason we are getting an
ARPANET PSN is because when DCA came out to do a site survey, they
liked our site so much they asked if they could put one here! It's
amazing how many sites have tried to get ARPANET PSN's the right
way and have had to wait much longer than us... BTW, since we are
dual homed (probably a gateway with 2 1822 interfaces in it), we
are taking steps to be sure that people on ARPANET or MILNET can't
use our gateway to bypass the mailbridges. The code will be hacked
to drop all packets that aren't going to a locally reachable network.
BARRNet, even though its locally reachable, will be excluded
from this however, since the current procedural limitations call for
not allowing any BARRNet traffic to flow out of BARRNet to MILNET
and the reverse. NASA traffic of course can traffic through BARRNet,
and even use ARPANET that way (though that's not a big deal when
we get our own ARPANET PSN). That's because only NASA is authorized
to directly connect to MILNET, not UCB or Stanford, etc...
DCA must have the ability to partition the ARPANET and MILNET in
case of an "emergency", and having non-DCA controlled paths between
the nets prevents that. There was talk some time ago about putting
explosive bolts in the mailbridges that would be triggered by
destruct packets... That idea didn't get far though...
The DDN only includes MILNET,ARPANET,SCINET,etc... Not the attached
networks. If it did, you'd need to file a TSR to add a PC to your
local cable. A TSR is a monstrous piece of paperwork that needs to
be done anytime anything is changed on the DDN... Rick knows all
about them don't you Rick?
The whole network game is filled with acronyms! I gave up trying
to write documents with full explainations in terms long ago...
I have yet to see a short and concise (and correct) way of describing
DDN X.25 Standard Service for example... That's probably one of the
harder things about getting into networking these days. We won't
even talk about Etherbunnies and Martians and other Millspeak...
Milo '1822' Medin
There were rumored to be "explosive bolts" on the ARPA/MILNET gateways (whether they were metaphorical or not, I don't know).
Here's something interesting that Milo Medin wrote about dual homed sites like NSA and NASA, that were on both the ARPANET and MILNET:
To: fair@ucbarpa.berkeley.edu (Erik E. Fair)
Cc: Hackers_Guild@ucbvax.berkeley.edu, ucdavis!ccohesh@ucbvax.berkeley.edu
Subject: Re: a question of definition
Date: Thu, 29 Jan 87 15:33:35 PST
From: Milo S. Medin (NASA ARC Code ED) <medin@orion.arpa>
Right, the core has many gateways on it now, maybe 20-30. All the LSI's will
be stubbed off the core however, and only buttergates will be left after
the mailbridges and EGP peers are all converted. Actually, I think DARPA is
paying for it all...
Ames is *not* getting a mailbridge. You are right of course, that we could
use 2 gateways, not just 1 (actually, there will be a prime and backup anyways),
and then push routing info appropriately. But that's anything but simple.
Firstly, the hosts have to know which gateway to send a packet to a given
network, and thus have to pick between the 2. That's a bad idea.
It also means that I have to pass all EGP learned info around on the
local cable, and if I do that, then I can't have routing info from
the local cable pass out via EGP. At least not without violating
the current EGP spec. Think about it. It'd be really simple to
create a loop that way. Thus, in order to maximize the use of both
PSN's, you really need one gateway wired to both PSN's, and just
have it advertise a default route inside. Or use a reasonble IGP,
of which RIP (aka /etc/routed stuff) is not. I'm hoping to get
an RFC out of BBN at this IETF meeting which may go a long way in
reducing the use of RIP as an IGP.
BTW, NSA is an example of a site on both MILNET and ARPANET but without
a mailbridge...
There is no restriction that a network can only be on ARPANET or MILNET.
That goes against the Internet model of doing things. Our local
NASA gatewayed nets will be advertised on both sides. The restriction
on BARRNet is that the constituent elements of BARRNet do not all
have access to MILNET. NSF has an understanding with DARPA and
DCA that NSFnet'd sites can use ARPANET. That does not extend to
the MILNET. Thus, Davis can use UCB's or Stanford's, our even NASA's
ARPANET gateways, with the approval of the site of course, but
not MILNET, even though NASA has MILNET coverage. Thus we are required
to restrict BARRNet routing through our MILNET PSN. If we were willing
to sponsor UCB's MILNET access, for some requirement which NASA
had to implement, then we would turn that on. But BARRNet itself will
but cutoff to MILNET (and probably ARPANET too) at Ames, but not
cut off to other NASA centers or sites that NASA connects. There is
no technical reason that prevents this, in fact, we have to take
special measures to prevent it. But those are the rules. Anyways,
mailbridge performance should improve after the conversion, so
UCB should be in better shape. And you'll certainly be able to
talk to us via BARRNnet... I have noticed recently that MILNET<->
ARPANET performance has been particularly poor... Sigh.
The DCA folks feel that in case of an emergency they may be
forced to use an unsecure network to pass certain info around. The
DDN brochure mentions SIOP related data for example. Who knows,
if the balloon goes up, the launch order might pass through Evans
Hall on its way out to SAC... :-)
Milo
Here's another funny story from my email archives of around the same time, about how Jordan Hubbard's infamous rwall almost got UC Berkeley cut off from the internet, with some more interesting details from other old net boys like Milo Medin, Marc Crispin, and Dennis G. Perry:
Speaking of YP (which I always thought sounded like a brand of moist baby poop towelettes), BSD, wildcard groups, SunRPC, and Sun's ingenuous networking and security and remote procedure call infrastructure, who remembers Jordan Hubbard's infamous rwall incident on March 31, 1987?
>On March 31, 1987 Hubbard executed an rwall command expecting it to send a message to every machine on the network at University of California, Berkeley, where he headed the Distributed Unix Group. The command instead began broadcasting Hubbard's message to every machine on the internet and was stopped after Hubbard realised the message was being broadcast remotely after he received complaints from people at Purdue University and University of Texas. Even though the command was terminated, it resulted in Hubbard receiving 743 messages and complaints, including one from the Inspector General of ARPAnet.
I was logged in on my Sun workstation "tumtum" when it happened, so I received his rwall too, and immediately sent him a humorous email with the subject of "flame flame flame" which I've lost in the intervening 35 years, but I still have a copy of his quick reply:
From: Jordan K. Hubbard <jkh%violet.Berkeley.EDU@berkeley.edu>
Date: Tue, Mar 31, 1987, 11:02 PM
To: Don Hopkins <don@tumtum.cs.umd.edu>
Subject: re: flame flame flame
Thanks, you were nicer than most.. Here's the stock letter I've been
sending back to people:
Thank you, thank you..
Now if I can only figure out why a lowly machine in a basement somewhere
can send broadcast messages to the entire world. Doesn't seem *right*
somehow.
Yours for an annoying network.
Jordan
P.S. I was actually experimenting to see exactly now bad a crock RPC was.
I'm beginning to get an idea. I look forward to your flame.
Jordan
Here's the explanation he sent to hackers_guild, and some replies from old net boys like Milo Medin (who said the program manager of the Arpanet in the Information Science and Technology Office of DARPA Dennis G. Perry said they would kick UCB off the Arpanet if it ever happened again), Mark Crispin (who presciently proposed cash rewards for discovering and disclosing security bugs), and Dennis G. Perry himself:
From: Jordan K. Hubbard <jkh%violet.Berkeley.EDU@berkeley.edu>
Date: April 2, 1987
Subject: My Broadcast
By now, many of you have heard of (or seen) the broadcast message I sent to
the net two days ago. I have since received 743 messages and have
replied to every one (either with a form letter, or more personally
when questions were asked). The intention behind this effort was to
show that I wasn't interested in doing what I did maliciously or in
hiding out afterwards and avoiding the repercussions. One of the
people who received my message was Dennis Perry, the Inspector General
of the ARPAnet (in the Pentagon), and he wasn't exactly pleased.
(I hear his Interleaf windows got scribbled on)
So now everyone is asking: "Who is this Jordan Hubbard, and why is he on my
screen??"
I will attempt to explain.
I head a small group here at Berkeley called the "Distributed Unix Group".
What that essentially means is that I come up with Unix distribution software
for workstations on campus. Part of this job entails seeing where some of
the novice administrators we're creating will hang themselves, and hopefully
prevent them from doing so. Yesterday, I finally got around to looking
at the "broadcast" group in /etc/netgroup which was set to "(,,)". It
was obvious that this was set up for rwall to use, so I read the documentation
on "netgroup" and "rwall". A section of the netgroup man page said:
...
Any of three fields can be empty, in which case it signifies
a wild card. Thus
universal (,,)
defines a group to which everyone belongs. Field names that ...
...
Now "everyone" here is pretty ambiguous. Reading a bit further down, one
sees discussion on yellow-pages domains and might be led to believe that
"everyone" was everyone in your domain. I know that rwall uses point-to-point
RPC connections, so I didn't feel that this was what they meant, just that
it seemed to be the implication.
Reading the rwall man page turned up nothing about "broadcasts". It doesn't
even specify the communications method used. One might infer that rwall
did indeed use actual broadcast packets.
Failing to find anything that might suggest that rwall would do anything
nasty beyond the bounds of the current domain (or at least up to the IMP),
I tried it. I knew that rwall takes awhile to do its stuff, so I left
it running and went back to my office. I assumed that anyone who got my
message would let me know.. Boy, was I right about that!
After the first few mail messages arrived from Purdue and Utexas, I begin
to understand what was really going on and killed the rwall. I mean, how
often do you expect to run something on your machine and have people
from Wisconsin start getting the results of it on their screens?
All of this has raised some interesting points and problems.
1. Rwall will walk through your entire hosts file and blare at anyone
and everyone if you use the (,,) wildcard group. Whether this is a bug
or a feature, I don't know.
2. Since rwall is an RPC service, and RPC doesn't seem to give a damn
who you are as long as you're root (which is trivial to be, on a work-
station), I have to wonder what other RPC services are open holes. We've
managed to do some interesting, unauthorized, things with the YP service
here at Berkeley, I wonder what the implications of this are.
3. Having a group called "broadcast" in your netgroup file (which is how
it comes from sun) is just begging for some novice admin (or operator
with root) to use it in the mistaken belief that he/she is getting to
all the users. I am really surprised (as are many others) that this has
taken this long to happen.
4. Killing rwall is not going to solve the problem. Any fool can write
rwall, and just about any fool can get root priviledge on a Sun workstation.
It seems that the place to fix the problem is on the receiving ends. The
only other alternative would be to tighten up all the IMP gateways to
forward packets only from "trusted" hosts. I don't like that at all,
from a standpoint of reduced convenience and productivity. Also, since
many places are adding hosts at a phenominal rate (ourselves especially),
it would be hard to keep such a database up to date. Many perfectly well-
behaved people would suffer for the potential sins of a few.
I certainly don't intend to do this again, but I'm very curious as to
what will happen as a result. A lot of people got wall'd, and I would think
that they would be annoyed that their machine would let someone from the
opposite side of the continent do such a thing!
Jordan Hubbard
jkh@violet.berkeley.edu (ucbvax!jkh)
Computer Facilities & Communications.
U.C. Berkeley
From: Milo S. Medin <medin@orion.arpa>
Date: Apr 6, 1987, 5:06 AM
Actually, Dennis Perry is the head of DARPA/IPTO, not a pencil pusher
in the IG's office. IPTO is the part of DARPA that deals with all
CS issues (including funding for ARPANET, BSD, MACH, SDINET, etc...).
Calling him part of the IG's office on the TCP/IP list probably didn't
win you any favors. Coincidentally I was at a meeting at the Pentagon
last Thursday that Dennis was at, along with Mike Corrigan (the man
at DoD/OSD responsible for all of DDN), and a couple other such types
discussing Internet management issues, when your little incident
came up. Dennis was absolutely livid, and I recall him saying something
about shutting off UCB's PSN ports if this happened again. There were
also reports about the DCA management types really putting on the heat
about turning on Mailbridge filtering now and not after the buttergates
are deployed. I don't know if Mike St. Johns and company can hold them
off much longer. Sigh... Mike Corrigan mentioned that this was the sort
of thing that gets networks shut off. You really pissed off the wrong
people with this move!
Dennis also called up some VP at SUN and demanded this hole
be patched in the next release. People generally pay attention
to such people.
Milo
From: Mark Crispin <MRC%PANDA@sumex-aim.stanford.edu>
Date: Mon, Apr 6, 1987, 10:15 AM
Dan -
I'm afraid you (and I, and any of the other old-timers who
care about security) are banging your head against a brick wall.
The philsophy behind Unix largely seems quite reminiscent of the
old ITS philsophy of "security through obscurity;" we must
entrust our systems and data to a open-ended set of youthful
hackers (the current term is "gurus") who have mastered the
arcane knowledge.
The problem is further exacerbated by the multitude of slimy
vendors who sell Unix boxes without sources and without an
efficient means of dealing with security problems as they
develop.
I don't see any relief, however. There are a lot of
politics involved here. Some individuals would rather muzzle
knowledge of Unix security problems and their fixes than see them
fixed. I feel it is *criminal* to have this attitude on the DDN,
since our national security in wartime might ultimately depend
upon it. If there is such a breach, those individuals will be
better off if the Russians win the war, because if not there will
be a Court of Inquiry to answer...
It may be necessary to take matters into our own hands, as
you did once before. I am seriously considering offering a cash
reward for the first discoverer of a Unix security bug, provided
that the bug is thoroughly documented (with both cause and fix).
There would be a sliding cash scale based on how devastating the
bug is and how many vendors' systems it affects. My intention
would be to propagate the knowledge as widely as possible with
the express intension of getting these bugs FIXED everywhere.
Knowledge is power, and it properly belongs in the hands of
system administrators and system programmers. It should NOT be
the exclusive province of "gurus" who have a vested interest in
keeping such details secret.
-- Mark --
PS: Crispin's definition of a "somewhat secure operating system":
A "somewhat secure operating system" is one that, given an
intelligent system management that does not commit a blunder that
compromises security, would withstand an attack by one of its
architects for at least an hour.
Crispin's definition of a "moderately secure operating system": a
"moderately secure operating system" is one that would withstand
an attack by one of its architects for at least an hour even if
the management of the system are total idiots who make every
mistake in the book.
-------
From: Dennis G. Perry <PERRY@vax.darpa.mil>
Date: Apr 6, 1987, 3:19 PM
Jordan, you are right in your assumptions that people will get annoyed
that what happened was allowed to happen.
By the way, I am the program manager of the Arpanet in the Information
Science and Technology Office of DARPA, located in Roslin (Arlington), not
the Pentagon.
I would like suggestions as to what you, or anyone else, think should be
done to prevent such occurances in the furture. There are many drastic
choices one could make. Is there a reasonable one? Perhaps some one
from Sun could volunteer what there action will be in light of this
revelation. I certainly hope that the community can come up with a good
solution, because I know that when the problem gets solved from the top
the solutions will reflect their concerns.
Think about this situation and I think you will all agree that this is
a serious problem that could cripple the Arpanet and anyother net that
lets things like this happen without control.
dennis
-------
I found this handy how-to tutorial guide for "Talking to the Milnet NOC" and resetting the LH/DH, which was useful for guiding the NSA employee on the other end of the phone through fixing their end of the problem. What it doesn't mention is that the key box with the chase key was extremely easy to pick with a paperclip.
Who would answer the Milnet NOC's 24-hour phone was hit or miss: Some were more helpful and knowledgeable than others, others were quite uptight.
Once I told the guy who answered, "Hi, this is the University of Maryland. Our connection to the NSA IMP seems to be down." He barked back: "You can't say that on the telephone! Are you calling on a blue phone?" (I can't remember the exact color, except that it wasn't red: that I would have remembered). I said, "You can't say NSA??! This is a green phone, but there's a black phone in the other room that I could call you back on, but then I couldn't see the hardware." And he said "No, I mean a voice secure line!" I replied, "You do know that this is a university, don't you? We only have black and green phones."