IBM white paper on Opteron

Here's an interesting IBM white paper on its own Opteron systems and their
advantages over Xeons and Itaniums:

http://www5.pc.ibm.com/us/me.nsf/webdocs/White+Paper:32-64-bit+HPC+and+Cluster+Computing+with+the+Opteron-Based+IBM+eServer+325:English/$FILE/HPC-e325.pdf

Yousuf Khan

On Mon, 20 Oct 2003 18:24:12 GMT, "Yousuf Khan"
wrote:

Here's an interesting IBM white paper on its own Opteron systems and their
advantages over Xeons and Itaniums:

http://www5.pc.ibm.com/us/me.nsf/webdocs/White+Paper:32-64-bit+HPC+and+Cluster+Computing+with+the+Opteron-Based+IBM+eServer+325:English/$FILE/HPC-e325.pdf

Yousuf Khan

That is to say, their advantages and disadvantages relative to Xeon
and Itanium _systems_.

If you look at the pluses and minuses chart, you'll notice that the
Itanium system compares most directly with the IBM Power4: emphasis on
large memory, high availability, and reliability. The Itanium and
Power4 systems are targeted at a market that Opteron isn't going to
penetrate, and they are priced accordingly.

Lowest cost 32-bit computing still goes to Xeon. All the neat
advantages of the Opteron system fit into the who cares category.

Do you need 64-bit computing? If you need to address a really large
flat memory space, you do.

The huge advantage of Opteron, its HT interface and integrated memory
controller in place of a front-side bus, doesn't even show up in the
chart, because it's a really neat feature that doesn't show up in a
_system_.

If there were really good infrastructure to support HT without going
through a PCI-type bus, Opteron could be a killer for the people who
don't have to conceal the AMD label from their boss. Their isn't,
though, unless something comes out of Red Storm that will be available
to ordinary mortals.

As it is, Opteron is an awesome solution looking for a problem.

RM

Robert Myers wrote:

On Mon, 20 Oct 2003 18:24:12 GMT, "Yousuf Khan"
wrote:


Here's an interesting IBM white paper on its own Opteron systems and their
advantages over Xeons and Itaniums:

http://www5.pc.ibm.com/us/me.nsf/webdocs/White+Paper:32-64-bit+HPC+and+Cluster+Computing+with+the+Opteron-Based+IBM+eServer+325:English/$FILE/HPC-e325.pdf

Yousuf Khan


That is to say, their advantages and disadvantages relative to Xeon
and Itanium _systems_.

If you look at the pluses and minuses chart, you'll notice that the
Itanium system compares most directly with the IBM Power4: emphasis on
large memory, high availability, and reliability. The Itanium and
Power4 systems are targeted at a market that Opteron isn't going to
penetrate, and they are priced accordingly.

That is because IBM *chose* to make it that way.
IBM isn't offering anything more than a dualie in the way of
Opteron systems. If they decided to make 4-way or 8-way
Opteron systems, then they would have no problem stacking
those systems up against 4 or 8 way Itanics and perhaps even
against Power4's - especially since the Opteron architecture
allows it to scale up *much* better than does the Itanic
or Power4.





Lowest cost 32-bit computing still goes to Xeon.

Slightly lower costs for Xeons and the privilege of being locked
into a dead end architecture, versus paying a little more for Opterons
that outperform the Xeons and have a wide open future upgrade path.
Plus IBM's Xeons can only handle 8 GB, vs 16 GB for the Opterons -
and the Opterons don't have to **** around with PAE when there is
more than 4 GB. Seems like an easy decision.

All the neat
advantages of the Opteron system fit into the who cares category.

Do you need 64-bit computing? If you need to address a really large
flat memory space, you do.

The huge advantage of Opteron, its HT interface and integrated memory
controller in place of a front-side bus, doesn't even show up in the
chart, because it's a really neat feature that doesn't show up in a
_system_.

If there were really good infrastructure to support HT without going
through a PCI-type bus, Opteron could be a killer for the people who
don't have to conceal the AMD label from their boss. Their isn't,
though, unless something comes out of Red Storm that will be available
to ordinary mortals.

As it is, Opteron is an awesome solution looking for a problem.

RM

On Sat, 25 Oct 2003 22:11:20 -0600, Rob Stow
wrote:

Robert Myers wrote:

On Mon, 20 Oct 2003 18:24:12 GMT, "Yousuf Khan"
wrote:


Here's an interesting IBM white paper on its own Opteron systems and their
advantages over Xeons and Itaniums:

http://www5.pc.ibm.com/us/me.nsf/webdocs/White+Paper:32-64-bit+HPC+and+Cluster+Computing+with+the+Opteron-Based+IBM+eServer+325:English/$FILE/HPC-e325.pdf

Yousuf Khan


That is to say, their advantages and disadvantages relative to Xeon
and Itanium _systems_.

If you look at the pluses and minuses chart, you'll notice that the
Itanium system compares most directly with the IBM Power4: emphasis on
large memory, high availability, and reliability. The Itanium and
Power4 systems are targeted at a market that Opteron isn't going to
penetrate, and they are priced accordingly.

That is because IBM *chose* to make it that way.
IBM isn't offering anything more than a dualie in the way of
Opteron systems. If they decided to make 4-way or 8-way
Opteron systems, then they would have no problem stacking
those systems up against 4 or 8 way Itanics and perhaps even
against Power4's - especially since the Opteron architecture
allows it to scale up *much* better than does the Itanic
or Power4.

When IBM or any other tier-one vendor decides to put out an Opteron
server clearly aimed at the Enterprise computing market, I'll take it
seriously as the wave of the future.

Until then, Opteron is a me-too competitor for Xeon with an irrelevant
difference. It will put price and performance pressure on Xeon
(that's good), but it wouldn't cause me to buy AMD stock.

As to scalability, we'll just have to see. I'm sure that RedStorm
will stack up nicely against an SGI Altix, but I'm not sure the price
to performance ratio will be any more attractive.

Lowest cost 32-bit computing still goes to Xeon.

Slightly lower costs for Xeons and the privilege of being locked
into a dead end architecture, versus paying a little more for Opterons
that outperform the Xeons and have a wide open future upgrade path.
Plus IBM's Xeons can only handle 8 GB, vs 16 GB for the Opterons -
and the Opterons don't have to **** around with PAE when there is
more than 4 GB. Seems like an easy decision.

Wide open future upgrade path? Are you sure of that?

x86-64 will survive, if it survives, because of Linux and geek users
like me who are always interested in something different, not to
mention that, if they get the connectivity issues straightened out
(how do you get past the eight-way, which, so far we haven't even seen
implemented), Opteron looks potentially attractive as a building block
for clusters.

Mainline computing? Forget it, unless IBM sees the light and decides
that they've pushed the Power architecture as far as they can. I
don't see any signs of that happening.

RM

"Robert Myers" wrote in message
...
Slightly lower costs for Xeons and the privilege of being locked
into a dead end architecture, versus paying a little more for Opterons
that outperform the Xeons and have a wide open future upgrade path.
Plus IBM's Xeons can only handle 8 GB, vs 16 GB for the Opterons -
and the Opterons don't have to **** around with PAE when there is
more than 4 GB. Seems like an easy decision.

Wide open future upgrade path? Are you sure of that?

x86-64 will survive, if it survives, because of Linux and geek users
like me who are always interested in something different, not to
mention that, if they get the connectivity issues straightened out
(how do you get past the eight-way, which, so far we haven't even seen
implemented), Opteron looks potentially attractive as a building block
for clusters.

Mainline computing? Forget it, unless IBM sees the light and decides
that they've pushed the Power architecture as far as they can. I
don't see any signs of that happening.

Oh come on, Rome wasn't built in a day. First the complaint was that no
tier-ones were offering Opterons. Now IBM is, and so the complaint is that
it's not offering 4- & 8-way Opterons. Then I suppose Opteron won't be taken
seriously until it's offered in 32- & 64-way systems? It's obvious that the
2-way system is good enough to offer to a Japanese institute as a
super-cluster computer.

No, it won't be just geeks keeping Opteron alive, it's much too expensive
for geeks to maintain one at home and nurture it themselves. It's definitely
an enterprise server system. If it isn't an enterprise server, then how do
Xeons make it into the heart of enterprises with piddly little 2- & 4-way
systems?

Yousuf Khan

On Sun, 26 Oct 2003 15:23:05 GMT, "Yousuf Khan"
wrote:

"Robert Myers" wrote in message
.. .
Slightly lower costs for Xeons and the privilege of being locked
into a dead end architecture, versus paying a little more for Opterons
that outperform the Xeons and have a wide open future upgrade path.
Plus IBM's Xeons can only handle 8 GB, vs 16 GB for the Opterons -
and the Opterons don't have to **** around with PAE when there is
more than 4 GB. Seems like an easy decision.

Wide open future upgrade path? Are you sure of that?

x86-64 will survive, if it survives, because of Linux and geek users
like me who are always interested in something different, not to
mention that, if they get the connectivity issues straightened out
(how do you get past the eight-way, which, so far we haven't even seen
implemented), Opteron looks potentially attractive as a building block
for clusters.

Mainline computing? Forget it, unless IBM sees the light and decides
that they've pushed the Power architecture as far as they can. I
don't see any signs of that happening.

Oh come on, Rome wasn't built in a day. First the complaint was that no
tier-ones were offering Opterons. Now IBM is, and so the complaint is that
it's not offering 4- & 8-way Opterons. Then I suppose Opteron won't be taken
seriously until it's offered in 32- & 64-way systems? It's obvious that the
2-way system is good enough to offer to a Japanese institute as a
super-cluster computer.

Japanese Institute = Geek user. Geek user = someone who knows enough
to judge the technology absolutely on its own merits, not worry all
that much about how well it will be supported, and find out what the
thing is really good for, not what the marketeers say it is good for.

But Japanese Institute != Enterprise user. Enterprise user = someone
who might have the inhouse resources to judge technology on its own
merits but doesn't trust them that much, worries most of all how well
the product will be supported, and wants to hear promises from someone
who can be sued if the promises aren't kept. IBM is a reliable maker
of promises, but the point of my post was that they aren't offering
Opteron as an Enterprise Server.

No, it won't be just geeks keeping Opteron alive, it's much too expensive
for geeks to maintain one at home and nurture it themselves.

Geek users != home users. The store near MIT says they're flying out
the door. Everybody gotta have one. Those are the people who built
_Gnu_/Linux, and those are the people who will guarantee that there is
at least some kind of future for Opteron.

It's definitely
an enterprise server system. If it isn't an enterprise server, then how do
Xeons make it into the heart of enterprises with piddly little 2- & 4-way
systems?

The IBM label is a great start. IBM has all kinds of reasons for
making the offering, not the least of which is that it was their
know-how that finally got the processor out the door. The IBM label
doesn't make it a candidate for an enterprise server. As I said, when
IBM or some other credible purveyor to companies with idiot boards of
directors starts marketing Opteron for large scale, mission critical
applications, I'll start taking it seriously as the wave of the
future.

If I keep this up long enough, Keith will be coming after me waving a
piece of wood that he insists is not a baseball bat and calling me an
idiot. Being an idiot actually isn't all that bad for judging a
situation like this, because it's how idiots will look at the
technology that will determine its future.

To return to the IBM document, *three* checkmarks for inexpensive
64-bit computing. Wow! But who needs inexpensive 64-bit computing?
Only an idiot would be impressed.

RM

"Robert Myers" wrote in message
...
On Sun, 26 Oct 2003 15:23:05 GMT, "Yousuf Khan"
wrote:
Oh come on, Rome wasn't built in a day. First the complaint was that no
tier-ones were offering Opterons. Now IBM is, and so the complaint is
that
it's not offering 4- & 8-way Opterons. Then I suppose Opteron won't be
taken
seriously until it's offered in 32- & 64-way systems? It's obvious that
the
2-way system is good enough to offer to a Japanese institute as a
super-cluster computer.

Japanese Institute = Geek user. Geek user = someone who knows enough
to judge the technology absolutely on its own merits, not worry all
that much about how well it will be supported, and find out what the
thing is really good for, not what the marketeers say it is good for.

But Japanese Institute != Enterprise user. Enterprise user = someone
who might have the inhouse resources to judge technology on its own
merits but doesn't trust them that much, worries most of all how well
the product will be supported, and wants to hear promises from someone
who can be sued if the promises aren't kept. IBM is a reliable maker
of promises, but the point of my post was that they aren't offering
Opteron as an Enterprise Server.

So what you seem to be implying is that IBM won't bother supporting these
Opteron servers as well as they support their "enterprise" servers. Perhaps
they have a special "geek"-level support program. Each server that you buy
comes with a year of "bronze geek" support, which you can upgrade and renew
to "silver" or "gold geek" at any time. But because it's the "geek" program
instead of an "enterprise" program, you'd be expected to diagnose your own
problems and change out all of your own parts. :-)

No, it won't be just geeks keeping Opteron alive, it's much too expensive
for geeks to maintain one at home and nurture it themselves.

Geek users != home users. The store near MIT says they're flying out
the door. Everybody gotta have one. Those are the people who built
_Gnu_/Linux, and those are the people who will guarantee that there is
at least some kind of future for Opteron.

Are these MIT users something akin to the "liberal Hollywood elites" in the
world of politics, except in this case the server computing world? You know
the folks who are just a little loopy and out on the fringe, and not
representative of true middle American/Server Computing values?

It's definitely
an enterprise server system. If it isn't an enterprise server, then how
do
Xeons make it into the heart of enterprises with piddly little 2- & 4-way
systems?

The IBM label is a great start. IBM has all kinds of reasons for
making the offering, not the least of which is that it was their
know-how that finally got the processor out the door.

That was the IBM Microelectronics folks that helped with the process
technology. As has been pointed out many times, the various IBM units don't
talk to each other, nor care what each other is doing, let alone help each
other's projects out. IBM's software division brought out a DB2 database for
Opteron, but that had no bearing on whether their server division brought
out a server based on it. It took some convincing by customers to this
division to get them to bring out a server based on the Opteron, not by
pressure from IBM's software or chipmaking units.

IBM used to make a lot of Cyrix processors at one time too, but none of
their PCs ever used them at the time.

The IBM label
doesn't make it a candidate for an enterprise server. As I said, when
IBM or some other credible purveyor to companies with idiot boards of
directors starts marketing Opteron for large scale, mission critical
applications, I'll start taking it seriously as the wave of the
future.

I kind of always assumed that a supercomputing cluster was considered a
large-scale, mission critical application.

To return to the IBM document, *three* checkmarks for inexpensive
64-bit computing. Wow! But who needs inexpensive 64-bit computing?
Only an idiot would be impressed.

I guess the fact that it runs 32-bit just as well and as inexpensively
really doesn't make a difference.

Yousuf Khan

On Sun, 26 Oct 2003 21:45:17 GMT, "Yousuf Khan"
wrote:

snip

I kind of always assumed that a supercomputing cluster was considered a
large-scale, mission critical application.

Supercomputers are for geeks. You'll never know, and I would expect
the information is classified, how much downtime DOE computers
experience, but it would be too much for the NYSE or even for UPS.

Because of the way government procurement works, we might wind up with
some Opterons in a truly mission-critical application, like air
traffic control. That still won't make Opteron an enterprise class
chip.

Now, it may be that AMD will somehow crack the vault and that x86-64
is the future of enterprise computing, but I don't think it is.

To return to the IBM document, *three* checkmarks for inexpensive
64-bit computing. Wow! But who needs inexpensive 64-bit computing?
Only an idiot would be impressed.

I guess the fact that it runs 32-bit just as well and as inexpensively
really doesn't make a difference.

But where is the clear advantage? Even Intel is having a tough time
cracking the enterprise market, but eventually it will, and with IA-64
leading the way. That's a safe bet. If you think it's not a safe
bet, look very deeply into your reasons for thinking so and invest
accordingly. You would win big if your reasoning were correct.

The compelling logic behind the processor market is the enormous cost
of developing them. There isn't room for many players. If you're a
big business, and you're making bets for the long-term future, it's a
toss-up right now as to which of two bets is the safer. On IBM's
side, they support backward compatibility in a way that makes Bill
Gates' whining about legacy seem a little pathetic. On Intel's side,
they have the cash flow, none of the baggage that IBM is carrying, and
alot of momentum. I don't see where AMD fits in.

But, you say, IBM makes computers and Intel makes chips. That is so,
but Intel doesn't seem to have any trouble finding people who want to
build computers with their chips, and, if anything, the logical
development would be for IBM to continue making computers and to stop
making chips.

RM

"Robert Myers" wrote in message
...
Supercomputers are for geeks. You'll never know, and I would expect
the information is classified, how much downtime DOE computers
experience, but it would be too much for the NYSE or even for UPS.

That's why these are super-clusters, not just singular supercomputers. With
500 or a 1000 nodes working together, I suspect the downtime would make the
NYSE or UPS jealous. Even if a single node or several nodes went down there
are hundreds of others still working. Might delay calculation output by
about 0.2% or whatever.

Because of the way government procurement works, we might wind up with
some Opterons in a truly mission-critical application, like air
traffic control. That still won't make Opteron an enterprise class
chip.

So then, what will make it an enterprise class computer chip? When the
corporate CIO gets treated to sumptuous buffet and is given a box seat at
the Superbowl?

Now, it may be that AMD will somehow crack the vault and that x86-64
is the future of enterprise computing, but I don't think it is.

To return to the IBM document, *three* checkmarks for inexpensive
64-bit computing. Wow! But who needs inexpensive 64-bit computing?
Only an idiot would be impressed.

I guess the fact that it runs 32-bit just as well and as inexpensively
really doesn't make a difference.

But where is the clear advantage?

Well obviously the "clear advantage" was the fact that it could also run
64-bit code.

What else do you want to hear? Is the right answer, in your mind, "we got
World Series and Stanley Cup box seats too"?

Even Intel is having a tough time
cracking the enterprise market, but eventually it will, and with IA-64
leading the way. That's a safe bet. If you think it's not a safe
bet, look very deeply into your reasons for thinking so and invest
accordingly. You would win big if your reasoning were correct.

Is it a safe bet? What is your reasoning for believing that? What question
is Itanium specifically answering? You accuse Opteron of being an answer in
search of a question, so the same goes for Itanium. What's the compelling
question that Itanium is answering?

The compelling logic behind the processor market is the enormous cost
of developing them. There isn't room for many players.

It's only an enormous cost if you set out with the specific goal of making
something at an enormous cost. Intel did everything possible to make this
Itanium project cost enormously: an absurd number of registers, a completely
experimental instruction set, and way too many layers of huge caches. Maybe
Intel was actually trying to prove this point to all other competitors, that
it can outspend them, sort of like a gorilla beating its chest. Yet they
ended up with a chip that's not really a stellar performer compared to any
of its competition, which were developed more cost-effectively. Intel
could've spent a tenth of this cost and come up with a better solution.

But, you say, IBM makes computers and Intel makes chips. That is so,
but Intel doesn't seem to have any trouble finding people who want to
build computers with their chips, and, if anything, the logical
development would be for IBM to continue making computers and to stop
making chips.

It seems to me that Intel and AMD are having precisely the opposite problems
with getting their architectures adopted. Intel has many many vendors
willing to sell their Itanium and very few customers willing to take them.
HP is about the only one selling these in any quantity because they are
forcing their HP-UX and Tru64 customers to adopt it. AMD on the other hand
has customers begging vendors to sell them a box with their processors on
it, and very few vendors willing to make one. IBM being the only one that
caved in to their customers so far.

Yousuf Khan

On Mon, 27 Oct 2003 01:38:34 GMT, "Yousuf Khan"
wrote:

"Robert Myers" wrote in message
.. .

snip


So then, what will make it an enterprise class computer chip? When the
corporate CIO gets treated to sumptuous buffet and is given a box seat at
the Superbowl?

Don't know that there is enough margin left in any of this stuff for
that sort of thing, but you're on the right track.

The salesman just won't even offer the Opteron boxes. For one thing,
no one will stuff 64 or 128 Opterons into the kind of SMP box that
gets sold into those situations, anyway. IBM may offer a box with 64
or 128 Opterons, but it will be cluster in a rack. Those will no more
be enterprise servers than the clusters of Xeon that IBM currently
sells in a rack.

snip

But where is the clear advantage?

Well obviously the "clear advantage" was the fact that it could also run
64-bit code.

And how much of *that* do you think there is going to be? For all
intents and purposes, Opteron is an expensive Xeon to everyone other
than those customers who write and compile their own softwa geeks,
in other words.

What else do you want to hear? Is the right answer, in your mind, "we got
World Series and Stanley Cup box seats too"?

Nice touch. :-).

Even Intel is having a tough time
cracking the enterprise market, but eventually it will, and with IA-64
leading the way. That's a safe bet. If you think it's not a safe
bet, look very deeply into your reasons for thinking so and invest
accordingly. You would win big if your reasoning were correct.

Is it a safe bet? What is your reasoning for believing that? What question
is Itanium specifically answering? You accuse Opteron of being an answer in
search of a question, so the same goes for Itanium. What's the compelling
question that Itanium is answering?

I think I already answered that question in the P4EE thread. The
architecture and philosophy of the Itanium is the only way that I know
of to address the growing gap between processor and memory speed.

The real von Neumann bottleneck in those huge SMP boxes are a few
sought-after pieces of data that everybody wants and everybody has to
wait for. The goal is to get it there when you need it, get done with
it as quickly as possible, and to let somebody else have it.

There are some pieces of this that depend on things I just don't have
the time to follow in detail--how the processors communicate is as
important as the processors themselves. Opteron was designed with
connectivity in mind, but it's a NUMA architecture, and what someone
does or does not do to glue more than four of these things together in
an SMP box will matter a great deal.

Itanium exists in those configurations from two manufacturers already:
HP and SGI. I wonder if such a box is even a gleam in anyone's eye
for Opteron.

The compelling logic behind the processor market is the enormous cost
of developing them. There isn't room for many players.

It's only an enormous cost if you set out with the specific goal of making
something at an enormous cost. Intel did everything possible to make this
Itanium project cost enormously: an absurd number of registers, a completely
experimental instruction set, and way too many layers of huge caches. Maybe
Intel was actually trying to prove this point to all other competitors, that
it can outspend them, sort of like a gorilla beating its chest.

That is exactly what they were trying to do.

Yet they
ended up with a chip that's not really a stellar performer compared to any
of its competition, which were developed more cost-effectively. Intel
could've spent a tenth of this cost and come up with a better solution.

Woulda. Shoulda. Coulda. Only government bureaucrats think that
projects that involve quantum leaps in technology come in on time and
under budget. They only get to believe that because they, their
bosses, the administration, and congress all collude in moving the
goal posts whenever necessary. I mentioned the space shuttle
elsewhere. It should be a standout beacon example of wishful
thinking, bad planning, gross mismanagement, irresponsible
engineering, and a whole host of other things, but (out of a mixture
of national pride, deliberate amnesia, and the fact that people have
died) no one wants to suck it up and admit what a screw-up it was from
beginning to end.

By any reasonable standard of comparison, Intel has been pounded
mercilessly on an incredibly ambitious project that has actually gone
rather well.

But, you say, IBM makes computers and Intel makes chips. That is so,
but Intel doesn't seem to have any trouble finding people who want to
build computers with their chips, and, if anything, the logical
development would be for IBM to continue making computers and to stop
making chips.

It seems to me that Intel and AMD are having precisely the opposite problems
with getting their architectures adopted. Intel has many many vendors
willing to sell their Itanium and very few customers willing to take them.
HP is about the only one selling these in any quantity because they are
forcing their HP-UX and Tru64 customers to adopt it. AMD on the other hand
has customers begging vendors to sell them a box with their processors on
it, and very few vendors willing to make one. IBM being the only one that
caved in to their customers so far.

Well, yes. But do you know what? AMD has everyone who is willing to
building motherboards for Opteron. Not so for Itanium. If you're not
on Intel's A list, you need not apply.

Intel isn't pushing Itanium hard because they know the technology
still isn't there. That they have been able to keep it competitive
with OoO processors with static scheduling is nothing short of
amazing.

What Intel isn't emphasizing is that the key idea: keep all the
scheduling off the die, just doesn't work well enough. For reasons
that have been discussed elsewhere, some kind of run-time flexibility
is necessary. So they're goint to wind up with all the stuff intended
to support compiler-scheduled flexibility (like predicated execution),
*and* on-die scheduling hardware. If I didn't think the original idea
was so beautiful and right at its core, I'd be laughing myself silly.

Intel, though, can afford it.

RM