[9235] in linux-scsi channel archive
Performance gap between 2.2.14 and 2.4.0-test4 kernels
daemon@ATHENA.MIT.EDU (Gianluca Cecchi)
Mon Jul 24 01:07:31 2000
Message-ID: <003901bff52d$9fb3d700$15c21d97@riccardo>
From: "Gianluca Cecchi" <gianluca.cecchi@teras.it>
To: <linux-scsi@vger.rutgers.edu>
Date: Mon, 24 Jul 2000 07:11:28 +0200
MIME-Version: 1.0
Content-Type: multipart/alternative;
boundary="----=_NextPart_000_0036_01BFF53E.62D64410"
This is a multi-part message in MIME format.
------=_NextPart_000_0036_01BFF53E.62D64410
Content-Type: text/plain;
charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable
The system:
MB: Supermicro P6SBU (Adaptec 7890 on board)
CPU: 1 pentium III 500 MHz
Mem: 256Mb
1x9.1Gb IBM DNES-309170W disk on fast/se channel
4x18.2Gb IBM DNES-318350W on ultra2 channel
The 18.2 Gb disks are in raid0 software. Below the /etc/raidtab file:
raiddev /dev/md0
raid-level 0
nr-raid-disks 4
persistent-superblock 1
chunk-size 128
device /dev/sdb1
raid-disk 0
device /dev/sdc1
raid-disk 1
device /dev/sdd1
raid-disk 2
device /dev/sde1
raid-disk 3 =20
output of dmesg related to scsi conf (in 2.4.0-test4 boot):
md.c: sizeof(mdp_super_t) =3D 4096
(scsi0) <Adaptec AIC-7890/1 Ultra2 SCSI host adapter> found at PCI =
0/14/0
(scsi0) Wide Channel, SCSI ID=3D7, 32/255 SCBs
(scsi0) Downloading sequencer code... 392 instructions downloaded
scsi0 : Adaptec AHA274x/284x/294x (EISA/VLB/PCI-Fast SCSI) 5.2.1/5.2.0
<Adaptec AIC-7890/1 Ultra2 SCSI host adapter>
scsi : 1 host.
(scsi0:0:5:0) Synchronous at 10.0 Mbyte/sec, offset 15.
Vendor: SONY Model: SDT-9000 Rev: 0400
Type: Sequential-Access ANSI SCSI revision: 02
(scsi0:0:6:0) Synchronous at 40.0 Mbyte/sec, offset 31.
Vendor: IBM Model: DNES-309170W Rev: SA30
Type: Direct-Access ANSI SCSI revision: 03
Detected scsi disk sda at scsi0, channel 0, id 6, lun 0
(scsi0:0:8:0) Synchronous at 80.0 Mbyte/sec, offset 31.
Vendor: IBM Model: DNES-318350W Rev: SA30
Type: Direct-Access ANSI SCSI revision: 03
Detected scsi disk sdb at scsi0, channel 0, id 8, lun 0
(scsi0:0:9:0) Synchronous at 80.0 Mbyte/sec, offset 31.
Vendor: IBM Model: DNES-318350W Rev: SA30
Type: Direct-Access ANSI SCSI revision: 03
Detected scsi disk sdc at scsi0, channel 0, id 9, lun 0
(scsi0:0:10:0) Synchronous at 80.0 Mbyte/sec, offset 31.
Vendor: IBM Model: DNES-318350W Rev: SA30
Type: Direct-Access ANSI SCSI revision: 03 =
=20
Detected scsi disk sdd at scsi0, channel 0, id 10, lun 0
(scsi0:0:12:0) Synchronous at 80.0 Mbyte/sec, offset 31.
Vendor: IBM Model: DNES-318350W Rev: SA30
Type: Direct-Access ANSI SCSI revision: 03
Detected scsi disk sde at scsi0, channel 0, id 12, lun 0
scsi : detected 5 SCSI disks total.
SCSI device sda: hdwr sector=3D 512 bytes. Sectors=3D 17916240 [8748 MB] =
[8.7 GB]
Partition check:
sda: sda1 sda2 < sda5 sda6 sda7 >
SCSI device sdb: hdwr sector=3D 512 bytes. Sectors=3D 35843670 [17501 =
MB] [17.5 GB]
sdb: sdb1
SCSI device sdc: hdwr sector=3D 512 bytes. Sectors=3D 35843670 [17501 =
MB] [17.5 GB]
sdc: sdc1
SCSI device sdd: hdwr sector=3D 512 bytes. Sectors=3D 35843670 [17501 =
MB] [17.5 GB]
sdd: sdd1
SCSI device sde: hdwr sector=3D 512 bytes. Sectors=3D 35843670 [17501 =
MB] [17.5 GB]
sde: sde1 =
=20
[snipped]
(read) sdb1's sb offset: 17920384 [events: 00000085]
(read) sdc1's sb offset: 17920384 [events: 00000085]
(read) sdd1's sb offset: 17920384 [events: 00000085]
(read) sde1's sb offset: 17920384 [events: 00000085]
autorun ...
considering sde1 ...
adding sde1 ...
adding sdd1 ...
adding sdc1 ...
adding sdb1 ...
created md0
bind<sdb1,1>
bind<sdc1,2>
bind<sdd1,3>
bind<sde1,4>
running: <sde1><sdd1><sdc1><sdb1>
now!
sde1's event counter: 00000085
sdd1's event counter: 00000085
sdc1's event counter: 00000085
sdb1's event counter: 00000085
raid0 personality registered
md0: max total readahead window set to 2048k
md0: 4 data-disks, max readahead per data-disk: 512k
raid0: looking at sdb1
raid0: comparing sdb1(17920384) with sdb1(17920384)
raid0: END
raid0: =3D=3D> UNIQUE
raid0: 1 zones
raid0: looking at sdc1
raid0: comparing sdc1(17920384) with sdb1(17920384)
raid0: EQUAL
raid0: looking at sdd1
raid0: comparing sdd1(17920384) with sdb1(17920384)
raid0: EQUAL
raid0: looking at sde1
raid0: comparing sde1(17920384) with sdb1(17920384)
raid0: EQUAL
raid0: FINAL 1 zones
zone 0
checking sdb1 ... contained as device 0
(17920384) is smallest!.
checking sdc1 ... contained as device 1
checking sdd1 ... contained as device 2
checking sde1 ... contained as device 3
zone->nb_dev: 4, size: 71681536
current zone offset: 17920384
done.
raid0 : md_size is 71681536 blocks.
raid0 : conf->smallest->size is 71681536 blocks.
raid0 : nb_zone is 1.
raid0 : Allocating 8 bytes for hash.
md: updating md0 RAID superblock on device
sde1 [events: 00000086](write) sde1's sb offset: 17920384
sdd1 [events: 00000086](write) sdd1's sb offset: 17920384
sdc1 [events: 00000086](write) sdc1's sb offset: 17920384
sdb1 [events: 00000086](write) sdb1's sb offset: 17920384=20
.
... autorun DONE.
Detected scsi tape st0 at scsi0, channel 0, id 5, lun 0
st: bufsize 32768, wrt 30720, max init. buffers 4, s/g segs 16. =
=20
These are the outputs of bonnie++ version 1.00 compiled on 2.2.14 kernel =
( redhat 6.2)
kernel 2.2.14 no tagged
Version 1.00 ------Sequential Output------ --Sequential Input- =
--Random-
-Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- =
--Seeks--
Machine MB K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP =
/sec %CP
Unknown 1000 8400 97 56908 70 21380 48 8475 96 58199 44 =
nan -21474836
48
------Sequential Create------ --------Random =
Create--------
-Create-- --Read--- -Delete-- -Create-- --Read--- =
-Delete--
files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP =
/sec %CP
30 175 96 627 99 6451 99 182 99 806 99 =
722 91
Unknown,1000,8400,97,56908,70,21380,48,8475,96,58199,44, =
nan,-2147483648,30,175,96,62
7,99,6451,99,182,99,806,99,722,91=20
kernel 2.4.0-test4 no tagged
Version 1.00 ------Sequential Output------ --Sequential Input- =
--Random-
-Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- =
--Seeks--
Machine MB K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP =
/sec %CP
Unknown 1000 8249 97 51642 37 10498 18 6190 72 17248 19 =
nan -2147483648
------Sequential Create------ --------Random =
Create--------
-Create-- --Read--- -Delete-- -Create-- --Read--- =
-Delete--
files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP =
/sec %CP
30 174 99 +++++ 93 9417 93 180 99 +++++ 104 =
1282 98
Unknown,1000,8249,97,51642,37,10498,18,6190,72,17248,19, =
nan,-2147483648,30,174,99,+++++,93,9417,93,180,99,+++++,104,1282,98 =
=20
=20
What about them? In particular the big difference between rewrite seq. =
output (21380 48 % cpu in 2.2 vs 10498 18%
in 2.4) and block seq. input (58199 44% in 2.2 vs 17248 19% in 2.4)?
Making dd or cp of big files the performances of 2.4 remains 1/3 respect =
of 2.2.14 (due to seq. input bad performance??),
eg. 62 secs for 512MB dd with 1MB block size versus 23 secs in 2.2.14). =
The cpu load is 45% in 2.2 versus 18% in 2.4.
The problem with kswapd overload seems not to be so present analyzing =
vmstat, but the=20
performance gap remains.
Is philosophical (logical) change toward multi user/multi processor =
environment or bad performance?=20
Thanks in advance for your clarifications.
Gianluca
PS: tell me if I can be of any help for testing conditions with my =
hardware.
------=_NextPart_000_0036_01BFF53E.62D64410
Content-Type: text/html;
charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN">
<HTML><HEAD>
<META content=3D"text/html; charset=3Diso-8859-1" =
http-equiv=3DContent-Type>
<META content=3D"MSHTML 5.00.2014.210" name=3DGENERATOR>
<STYLE></STYLE>
</HEAD>
<BODY bgColor=3D#ffffff>
<DIV> </DIV>
<DIV><FONT face=3DArial size=3D2>The system:</FONT></DIV>
<DIV> </DIV>
<DIV><FONT face=3DArial size=3D2>MB: Supermicro P6SBU (Adaptec 7890 on=20
board)<BR>CPU: 1 pentium III 500 MHz<BR>Mem: 256Mb</FONT></DIV>
<DIV> </DIV>
<DIV><FONT face=3DArial size=3D2>1x9.1Gb IBM DNES-309170W disk on =
fast/se=20
channel<BR>4x18.2Gb IBM DNES-318350W on ultra2 channel<BR>The 18.2 Gb =
disks are=20
in raid0 software. Below the /etc/raidtab file:</FONT></DIV>
<DIV> </DIV>
<DIV><FONT face=3DArial size=3D2>raiddev=20
/dev/md0<BR> =20
raid-level =20
0<BR> =20
nr-raid-disks =20
4<BR> =20
persistent-superblock=20
1<BR> =20
chunk-size =20
128<BR> =20
device =20
/dev/sdb1<BR> =20
raid-disk =20
0<BR> =20
device =20
/dev/sdc1<BR> =20
raid-disk =20
1<BR> =20
device =20
/dev/sdd1<BR> =20
raid-disk =20
2<BR> =20
device =20
/dev/sde1<BR> =20
raid-disk =20
3 =
=20
</FONT></DIV>
<DIV> </DIV>
<DIV><FONT face=3DArial size=3D2>output of dmesg related to scsi conf =
(in=20
2.4.0-test4 boot):</FONT></DIV>
<DIV> </DIV>
<DIV><FONT face=3DArial size=3D2>md.c: sizeof(mdp_super_t) =3D =
4096<BR>(scsi0)=20
<Adaptec AIC-7890/1 Ultra2 SCSI host adapter> found at PCI=20
0/14/0<BR>(scsi0) Wide Channel, SCSI ID=3D7, 32/255 SCBs<BR>(scsi0) =
Downloading=20
sequencer code... 392 instructions downloaded<BR>scsi0 : Adaptec=20
AHA274x/284x/294x (EISA/VLB/PCI-Fast SCSI)=20
5.2.1/5.2.0<BR> <Adaptec =
AIC-7890/1=20
Ultra2 SCSI host adapter><BR>scsi : 1 host.<BR>(scsi0:0:5:0) =
Synchronous at=20
10.0 Mbyte/sec, offset 15.<BR> Vendor: =
SONY =20
Model: SDT-9000 =
Rev:=20
0400<BR> Type: =20
Sequential-Access &n=
bsp; =20
ANSI SCSI revision: 02<BR>(scsi0:0:6:0) Synchronous at 40.0 Mbyte/sec, =
offset=20
31.<BR> Vendor: IBM Model:=20
DNES-309170W Rev: SA30<BR> =
Type: =20
Direct-Access =
=20
ANSI SCSI revision: 03<BR>Detected scsi disk sda at scsi0, channel 0, id =
6, lun=20
0<BR>(scsi0:0:8:0) Synchronous at 80.0 Mbyte/sec, offset 31.<BR> =
Vendor:=20
IBM Model:=20
DNES-318350W Rev: SA30<BR> =
Type: =20
Direct-Access =
=20
ANSI SCSI revision: 03<BR>Detected scsi disk sdb at scsi0, channel 0, id =
8, lun=20
0<BR>(scsi0:0:9:0) Synchronous at 80.0 Mbyte/sec, offset 31.<BR> =
Vendor:=20
IBM Model:=20
DNES-318350W Rev: SA30<BR> =
Type: =20
Direct-Access =
=20
ANSI SCSI revision: 03<BR>Detected scsi disk sdc at scsi0, channel 0, id =
9, lun=20
0<BR>(scsi0:0:10:0) Synchronous at 80.0 Mbyte/sec, offset 31.<BR> =
Vendor:=20
IBM Model:=20
DNES-318350W Rev: SA30<BR> =
Type: =20
Direct-Access =
=20
ANSI SCSI revision: 03 =
<BR>Detected=20
scsi disk sdd at scsi0, channel 0, id 10, lun 0<BR>(scsi0:0:12:0) =
Synchronous at=20
80.0 Mbyte/sec, offset 31.<BR> Vendor:=20
IBM Model:=20
DNES-318350W Rev: SA30<BR> =
Type: =20
Direct-Access =
=20
ANSI SCSI revision: 03<BR>Detected scsi disk sde at scsi0, channel 0, id =
12, lun=20
0<BR>scsi : detected 5 SCSI disks total.<BR>SCSI device sda: hdwr =
sector=3D 512=20
bytes. Sectors=3D 17916240 [8748 MB] [8.7 GB]<BR>Partition =
check:<BR> sda:=20
sda1 sda2 < sda5 sda6 sda7 ><BR>SCSI device sdb: hdwr sector=3D =
512 bytes.=20
Sectors=3D 35843670 [17501 MB] [17.5 GB]<BR> sdb: sdb1<BR>SCSI =
device sdc:=20
hdwr sector=3D 512 bytes. Sectors=3D 35843670 [17501 MB] [17.5 =
GB]<BR> sdc:=20
sdc1<BR>SCSI device sdd: hdwr sector=3D 512 bytes. Sectors=3D 35843670 =
[17501 MB]=20
[17.5 GB]<BR> sdd: sdd1<BR>SCSI device sde: hdwr sector=3D 512 =
bytes.=20
Sectors=3D 35843670 [17501 MB] [17.5 GB]<BR> sde:=20
sde1 &nb=
sp; &nbs=
p;  =
; =
&=
nbsp; =20
</FONT></DIV>
<DIV> </DIV>
<DIV><FONT face=3DArial size=3D2>[snipped]</FONT></DIV>
<DIV> </DIV>
<DIV><FONT face=3DArial size=3D2>(read) sdb1's sb offset: 17920384 =
[events:=20
00000085]<BR>(read) sdc1's sb offset: 17920384 [events: =
00000085]<BR>(read)=20
sdd1's sb offset: 17920384 [events: 00000085]<BR>(read) sde1's sb =
offset:=20
17920384 [events: 00000085]<BR>autorun ...<BR>considering sde1 =
...<BR> =20
adding sde1 ...<BR> adding sdd1 ...<BR> adding sdc1 =
...<BR> =20
adding sdb1 ...<BR>created=20
md0<BR>bind<sdb1,1><BR>bind<sdc1,2><BR>bind<sdd1,3><BR>=
bind<sde1,4><BR>running:=20
<sde1><sdd1><sdc1><sdb1><BR>now!<BR>sde1's event =
counter: 00000085<BR>sdd1's event counter: 00000085<BR>sdc1's event =
counter:=20
00000085<BR>sdb1's event counter: 00000085<BR>raid0 personality=20
registered<BR>md0: max total readahead window set to 2048k<BR>md0: 4 =
data-disks,=20
max readahead per data-disk: 512k<BR>raid0: looking at=20
sdb1<BR>raid0: comparing sdb1(17920384) with=20
sdb1(17920384)<BR>raid0: END<BR>raid0: =
=3D=3D>=20
UNIQUE<BR>raid0: 1 zones<BR>raid0: looking at sdc1<BR>raid0: =
comparing sdc1(17920384) with sdb1(17920384)<BR>raid0: =20
EQUAL<BR>raid0: looking at sdd1<BR>raid0: comparing =
sdd1(17920384)=20
with sdb1(17920384)<BR>raid0: EQUAL<BR>raid0: looking at=20
sde1<BR>raid0: comparing sde1(17920384) with=20
sdb1(17920384)<BR>raid0: EQUAL<BR>raid0: FINAL 1 =
zones<BR>zone=20
0<BR> checking sdb1 ... contained as device 0<BR> (17920384) =
is=20
smallest!.<BR> checking sdc1 ... contained as device =
1<BR> checking=20
sdd1 ... contained as device 2<BR> checking sde1 ... contained as =
device=20
3<BR> zone->nb_dev: 4, size: 71681536<BR>current zone offset:=20
17920384<BR>done.<BR>raid0 : md_size is 71681536 blocks.<BR>raid0 :=20
conf->smallest->size is 71681536 blocks.<BR>raid0 : nb_zone is =
1.<BR>raid0=20
: Allocating 8 bytes for hash.<BR>md: updating md0 RAID superblock on=20
device<BR>sde1 [events: 00000086](write) sde1's sb offset: =
17920384<BR>sdd1=20
[events: 00000086](write) sdd1's sb offset: 17920384<BR>sdc1 [events:=20
00000086](write) sdc1's sb offset: 17920384<BR>sdb1 [events: =
00000086](write)=20
sdb1's sb offset: 17920384 <BR>.<BR>... autorun DONE.<BR>Detected scsi =
tape st0=20
at scsi0, channel 0, id 5, lun 0<BR>st: bufsize 32768, wrt 30720, max =
init.=20
buffers 4, s/g segs=20
16. &nbs=
p; =20
</FONT></DIV>
<DIV> </DIV>
<DIV><FONT face=3DArial size=3D2><BR>These are the outputs of bonnie++ =
version 1.00=20
compiled on 2.2.14 kernel ( redhat 6.2)</FONT></DIV>
<DIV> </DIV>
<DIV><FONT face=3DArial size=3D2><BR>kernel 2.2.14 no =
tagged<BR>Version =20
1.00 ------Sequential Output------=20
--Sequential Input-=20
--Random-<BR> =
=20
-Per Chr- --Block-- -Rewrite- -Per Chr- --Block--=20
--Seeks--<BR>Machine  =
; MB=20
K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec=20
%CP<BR>Unknown 1000 =
8400 =20
97 56908 70 21380 48 8475 96 58199 =
44 =20
nan=20
-21474836<BR>48<BR> =
=20
------Sequential Create------ --------Random=20
Create--------<BR> &=
nbsp; =20
-Create-- --Read--- -Delete-- -Create-- --Read---=20
-Delete--<BR> =
=20
files /sec %CP /sec %CP /sec %CP /sec %CP =
/sec=20
%CP /sec=20
%CP<BR> =
=20
30 175 96 627 99 6451 =20
99 182 99 806 99 =
722 =20
91<BR>Unknown,1000,8400,97,56908,70,21380,48,8475,96,58199,44,=20
nan,-2147483648,30,175,96,62<BR>7,99,6451,99,182,99,806,99,722,91 =
</FONT></DIV>
<DIV> </DIV>
<DIV><FONT face=3DArial size=3D2>kernel 2.4.0-test4 no =
tagged<BR>Version =20
1.00 ------Sequential Output------=20
--Sequential Input-=20
--Random-<BR> =
=20
-Per Chr- --Block-- -Rewrite- -Per Chr- --Block--=20
--Seeks--<BR>Machine  =
; MB=20
K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec=20
%CP<BR>Unknown 1000 =
8249 =20
97 51642 37 10498 18 6190 72 17248 =
19 =20
nan=20
-2147483648<BR> &nbs=
p; =20
------Sequential Create------ --------Random=20
Create--------<BR> &=
nbsp; =20
-Create-- --Read--- -Delete-- -Create-- --Read---=20
-Delete--<BR> =
=20
files /sec %CP /sec %CP /sec %CP /sec %CP =
/sec=20
%CP /sec=20
%CP<BR> =
=20
30 174 99 +++++ 93 9417 =
93 =20
180 99 +++++ 104 1282 =20
98<BR>Unknown,1000,8249,97,51642,37,10498,18,6190,72,17248,19,=20
nan,-2147483648,30,174,99,+++++,93,9417,93,180,99,+++++,104,1282,98 =
&=
nbsp; =20
&=
nbsp; =20
</FONT></DIV>
<DIV><FONT face=3DArial size=3D2></FONT> </DIV>
<DIV><FONT face=3DArial size=3D2>What about them? In particular the big =
difference=20
between rewrite seq. output (21380 48 % cpu in 2.2 vs =
10498 =20
18%</FONT></DIV>
<DIV><FONT face=3DArial size=3D2>in 2.4) and block seq. input =
(58199 44% in 2.2 vs 17248 19% in 2.4)?</FONT></DIV>
<DIV> </DIV>
<DIV><FONT face=3DArial size=3D2>Making dd or cp of big files the =
performances of=20
2.4 remains 1/3 respect of 2.2.14 (due to seq. input bad=20
performance??),</FONT></DIV>
<DIV><FONT face=3DArial size=3D2>eg. 62 secs for 512MB dd with 1MB block =
size versus=20
23 secs in 2.2.14). The cpu load is 45% in 2.2 versus 18% in 2.4.<BR>The =
problem=20
with kswapd overload seems not to be so present analyzing vmstat, but =
the=20
<BR>performance gap remains.<BR>Is philosophical (logical) change toward =
multi=20
user/multi processor environment or bad performance? <BR>Thanks in =
advance for=20
your clarifications.<BR>Gianluca</FONT></DIV>
<DIV> </DIV>
<DIV><FONT face=3DArial size=3D2><BR>PS: tell me if I can be of any help =
for testing=20
conditions with my hardware.<BR></FONT></DIV></BODY></HTML>
------=_NextPart_000_0036_01BFF53E.62D64410--
-
To unsubscribe from this list: send the line "unsubscribe linux-scsi" in
the body of a message to majordomo@vger.rutgers.edu
