Estou trabalhando com 2 x chassis Supermicro SC837E26-RJBOD1 conectados a uma única placa LSI 9285-8e em um host Supermicro 1U. Existem 28 drives em cada chassi para um total de 56 drives em 28 espelhos RAID1.
O problema que estou correndo é que há slots duplicados para o chassi 2 (a lista de slots duas vezes e só vai de 0 a 27). Todas as unidades também mostram o mesmo ID de gabinete (ID 36). No entanto, MegaCLI -encinfo lista os dois gabinetes corretamente (ID 36 e ID 65).
Minha pergunta é: por que isso aconteceria? Existe uma opção que estou perdendo para usar 2 compartimentos de forma eficaz?
Isso está me impedindo de recriar uma unidade que falhou no slot 11, pois só posso especificar o gabinete e o slot como parâmetros para substituir uma unidade. Quando faço isso, ele escolhe o slot errado 11 (ID do dispositivo 46 em vez do ID do dispositivo 19).
O adaptador nº 1 é o LSI 9285-8e, o adaptador nº 0 (que removi devido a limitações de espaço) é o LSI integrado.
Informações do adaptador:
Adapter #1
==============================================================================
Versions
================
Product Name : LSI MegaRAID SAS 9285-8e
Serial No : SV12704804
FW Package Build: 23.1.1-0004
Mfg. Data
================
Mfg. Date : 06/30/11
Rework Date : 00/00/00
Revision No : 00A
Battery FRU : N/A
Image Versions in Flash:
================
BIOS Version : 5.25.00_4.11.05.00_0x05040000
WebBIOS Version : 6.1-20-e_20-Rel
Preboot CLI Version: 05.01-04:#%00001
FW Version : 3.140.15-1320
NVDATA Version : 2.1106.03-0051
Boot Block Version : 2.04.00.00-0001
BOOT Version : 06.253.57.219
Pending Images in Flash
================
None
PCI Info
================
Vendor Id : 1000
Device Id : 005b
SubVendorId : 1000
SubDeviceId : 9285
Host Interface : PCIE
ChipRevision : B0
Number of Frontend Port: 0
Device Interface : PCIE
Number of Backend Port: 8
Port : Address
0 5003048000ee8e7f
1 5003048000ee8a7f
2 0000000000000000
3 0000000000000000
4 0000000000000000
5 0000000000000000
6 0000000000000000
7 0000000000000000
HW Configuration
================
SAS Address : 500605b0038f9210
BBU : Present
Alarm : Present
NVRAM : Present
Serial Debugger : Present
Memory : Present
Flash : Present
Memory Size : 1024MB
TPM : Absent
On board Expander: Absent
Upgrade Key : Absent
Temperature sensor for ROC : Present
Temperature sensor for controller : Absent
ROC temperature : 70 degree Celcius
Settings
================
Current Time : 18:24:36 3/13, 2012
Predictive Fail Poll Interval : 300sec
Interrupt Throttle Active Count : 16
Interrupt Throttle Completion : 50us
Rebuild Rate : 30%
PR Rate : 30%
BGI Rate : 30%
Check Consistency Rate : 30%
Reconstruction Rate : 30%
Cache Flush Interval : 4s
Max Drives to Spinup at One Time : 2
Delay Among Spinup Groups : 12s
Physical Drive Coercion Mode : Disabled
Cluster Mode : Disabled
Alarm : Enabled
Auto Rebuild : Enabled
Battery Warning : Enabled
Ecc Bucket Size : 15
Ecc Bucket Leak Rate : 1440 Minutes
Restore HotSpare on Insertion : Disabled
Expose Enclosure Devices : Enabled
Maintain PD Fail History : Enabled
Host Request Reordering : Enabled
Auto Detect BackPlane Enabled : SGPIO/i2c SEP
Load Balance Mode : Auto
Use FDE Only : No
Security Key Assigned : No
Security Key Failed : No
Security Key Not Backedup : No
Default LD PowerSave Policy : Controller Defined
Maximum number of direct attached drives to spin up in 1 min : 10
Any Offline VD Cache Preserved : No
Allow Boot with Preserved Cache : No
Disable Online Controller Reset : No
PFK in NVRAM : No
Use disk activity for locate : No
Capabilities
================
RAID Level Supported : RAID0, RAID1, RAID5, RAID6, RAID00, RAID10, RAID50, RAID60, PRL 11, PRL 11 with spanning, SRL 3 supported, PRL11-RLQ0 DDF layout with no span, PRL11-RLQ0 DDF layout with span
Supported Drives : SAS, SATA
Allowed Mixing:
Mix in Enclosure Allowed
Mix of SAS/SATA of HDD type in VD Allowed
Status
================
ECC Bucket Count : 0
Limitations
================
Max Arms Per VD : 32
Max Spans Per VD : 8
Max Arrays : 128
Max Number of VDs : 64
Max Parallel Commands : 1008
Max SGE Count : 60
Max Data Transfer Size : 8192 sectors
Max Strips PerIO : 42
Max LD per array : 16
Min Strip Size : 8 KB
Max Strip Size : 1.0 MB
Max Configurable CacheCade Size: 0 GB
Current Size of CacheCade : 0 GB
Current Size of FW Cache : 887 MB
Device Present
================
Virtual Drives : 28
Degraded : 0
Offline : 0
Physical Devices : 59
Disks : 56
Critical Disks : 0
Failed Disks : 0
Supported Adapter Operations
================
Rebuild Rate : Yes
CC Rate : Yes
BGI Rate : Yes
Reconstruct Rate : Yes
Patrol Read Rate : Yes
Alarm Control : Yes
Cluster Support : No
BBU : No
Spanning : Yes
Dedicated Hot Spare : Yes
Revertible Hot Spares : Yes
Foreign Config Import : Yes
Self Diagnostic : Yes
Allow Mixed Redundancy on Array : No
Global Hot Spares : Yes
Deny SCSI Passthrough : No
Deny SMP Passthrough : No
Deny STP Passthrough : No
Support Security : No
Snapshot Enabled : No
Support the OCE without adding drives : Yes
Support PFK : Yes
Support PI : No
Support Boot Time PFK Change : Yes
Disable Online PFK Change : No
PFK TrailTime Remaining : 0 days 0 hours
Support Shield State : Yes
Block SSD Write Disk Cache Change: Yes
Supported VD Operations
================
Read Policy : Yes
Write Policy : Yes
IO Policy : Yes
Access Policy : Yes
Disk Cache Policy : Yes
Reconstruction : Yes
Deny Locate : No
Deny CC : No
Allow Ctrl Encryption: No
Enable LDBBM : No
Support Breakmirror : No
Power Savings : Yes
Supported PD Operations
================
Force Online : Yes
Force Offline : Yes
Force Rebuild : Yes
Deny Force Failed : No
Deny Force Good/Bad : No
Deny Missing Replace : No
Deny Clear : No
Deny Locate : No
Support Temperature : Yes
Disable Copyback : No
Enable JBOD : No
Enable Copyback on SMART : No
Enable Copyback to SSD on SMART Error : Yes
Enable SSD Patrol Read : No
PR Correct Unconfigured Areas : Yes
Enable Spin Down of UnConfigured Drives : Yes
Disable Spin Down of hot spares : No
Spin Down time : 30
T10 Power State : Yes
Error Counters
================
Memory Correctable Errors : 0
Memory Uncorrectable Errors : 0
Cluster Information
================
Cluster Permitted : No
Cluster Active : No
Default Settings
================
Phy Polarity : 0
Phy PolaritySplit : 0
Background Rate : 30
Strip Size : 64kB
Flush Time : 4 seconds
Write Policy : WB
Read Policy : Adaptive
Cache When BBU Bad : Disabled
Cached IO : No
SMART Mode : Mode 6
Alarm Disable : Yes
Coercion Mode : None
ZCR Config : Unknown
Dirty LED Shows Drive Activity : No
BIOS Continue on Error : No
Spin Down Mode : None
Allowed Device Type : SAS/SATA Mix
Allow Mix in Enclosure : Yes
Allow HDD SAS/SATA Mix in VD : Yes
Allow SSD SAS/SATA Mix in VD : No
Allow HDD/SSD Mix in VD : No
Allow SATA in Cluster : No
Max Chained Enclosures : 16
Disable Ctrl-R : Yes
Enable Web BIOS : Yes
Direct PD Mapping : No
BIOS Enumerate VDs : Yes
Restore Hot Spare on Insertion : No
Expose Enclosure Devices : Yes
Maintain PD Fail History : Yes
Disable Puncturing : No
Zero Based Enclosure Enumeration : No
PreBoot CLI Enabled : Yes
LED Show Drive Activity : Yes
Cluster Disable : Yes
SAS Disable : No
Auto Detect BackPlane Enable : SGPIO/i2c SEP
Use FDE Only : No
Enable Led Header : No
Delay during POST : 0
EnableCrashDump : No
Disable Online Controller Reset : No
EnableLDBBM : No
Un-Certified Hard Disk Drives : Allow
Treat Single span R1E as R10 : No
Max LD per array : 16
Power Saving option : Don't Auto spin down Configured Drives
Max power savings option is not allowed for LDs. Only T10 power conditions are to be used.
Default spin down time in minutes: 30
Enable JBOD : No
TTY Log In Flash : No
Auto Enhanced Import : No
BreakMirror RAID Support : No
Disable Join Mirror : No
Enable Shield State : Yes
Time taken to detect CME : 60s
Exit Code: 0x00
Informações sobre o recinto:
# /opt/MegaRAID/MegaCli/MegaCli64 -encinfo -a1
Number of enclosures on adapter 1 -- 3
Enclosure 0:
Device ID : 36
Number of Slots : 28
Number of Power Supplies : 2
Number of Fans : 3
Number of Temperature Sensors : 1
Number of Alarms : 1
Number of SIM Modules : 0
Number of Physical Drives : 28
Status : Normal
Position : 1
Connector Name : Port B
Enclosure type : SES
VendorId is LSI CORP and Product Id is SAS2X36
VendorID and Product ID didnt match
FRU Part Number : N/A
Enclosure Serial Number : N/A
ESM Serial Number : N/A
Enclosure Zoning Mode : N/A
Partner Device Id : 65
Inquiry data :
Vendor Identification : LSI CORP
Product Identification : SAS2X36
Product Revision Level : 0718
Vendor Specific : x36-55.7.24.1
Number of Voltage Sensors :2
Voltage Sensor :0
Voltage Sensor Status :OK
Voltage Value :5020 milli volts
Voltage Sensor :1
Voltage Sensor Status :OK
Voltage Value :11820 milli volts
Number of Power Supplies : 2
Power Supply : 0
Power Supply Status : OK
Power Supply : 1
Power Supply Status : OK
Number of Fans : 3
Fan : 0
Fan Speed :Low Speed
Fan Status : OK
Fan : 1
Fan Speed :Low Speed
Fan Status : OK
Fan : 2
Fan Speed :Low Speed
Fan Status : OK
Number of Temperature Sensors : 1
Temp Sensor : 0
Temperature : 48
Temperature Sensor Status : OK
Number of Chassis : 1
Chassis : 0
Chassis Status : OK
Enclosure 1:
Device ID : 65
Number of Slots : 28
Number of Power Supplies : 2
Number of Fans : 3
Number of Temperature Sensors : 1
Number of Alarms : 1
Number of SIM Modules : 0
Number of Physical Drives : 28
Status : Normal
Position : 1
Connector Name : Port A
Enclosure type : SES
VendorId is LSI CORP and Product Id is SAS2X36
VendorID and Product ID didnt match
FRU Part Number : N/A
Enclosure Serial Number : N/A
ESM Serial Number : N/A
Enclosure Zoning Mode : N/A
Partner Device Id : 36
Inquiry data :
Vendor Identification : LSI CORP
Product Identification : SAS2X36
Product Revision Level : 0718
Vendor Specific : x36-55.7.24.1
Number of Voltage Sensors :2
Voltage Sensor :0
Voltage Sensor Status :OK
Voltage Value :5020 milli volts
Voltage Sensor :1
Voltage Sensor Status :OK
Voltage Value :11760 milli volts
Number of Power Supplies : 2
Power Supply : 0
Power Supply Status : OK
Power Supply : 1
Power Supply Status : OK
Number of Fans : 3
Fan : 0
Fan Speed :Low Speed
Fan Status : OK
Fan : 1
Fan Speed :Low Speed
Fan Status : OK
Fan : 2
Fan Speed :Low Speed
Fan Status : OK
Number of Temperature Sensors : 1
Temp Sensor : 0
Temperature : 47
Temperature Sensor Status : OK
Number of Chassis : 1
Chassis : 0
Chassis Status : OK
Enclosure 2:
Device ID : 252
Number of Slots : 8
Number of Power Supplies : 0
Number of Fans : 0
Number of Temperature Sensors : 0
Number of Alarms : 0
Number of SIM Modules : 1
Number of Physical Drives : 0
Status : Normal
Position : 1
Connector Name : Unavailable
Enclosure type : SGPIO
Failed in first Inquiry commnad
FRU Part Number : N/A
Enclosure Serial Number : N/A
ESM Serial Number : N/A
Enclosure Zoning Mode : N/A
Partner Device Id : Unavailable
Inquiry data :
Vendor Identification : LSI
Product Identification : SGPIO
Product Revision Level : N/A
Vendor Specific :
Exit Code: 0x00
Agora, observe que cada dispositivo de slot 11 mostra um ID de 36, acho que é onde a discrepância acontece. Um deve ser 36. Mas o outro deve estar no gabinete 65.
Drives no espaço 11:
Enclosure Device ID: 36
Slot Number: 11
Drive's postion: DiskGroup: 5, Span: 0, Arm: 1
Enclosure position: 0
Device Id: 48
WWN:
Sequence Number: 11
Media Error Count: 0
Other Error Count: 0
Predictive Failure Count: 0
Last Predictive Failure Event Seq Number: 0
PD Type: SATA
Raw Size: 2.728 TB [0x15d50a3b0 Sectors]
Non Coerced Size: 2.728 TB [0x15d40a3b0 Sectors]
Coerced Size: 2.728 TB [0x15d400000 Sectors]
Firmware state: Online, Spun Up
Is Commissioned Spare : YES
Device Firmware Level: A5C0
Shield Counter: 0
Successful diagnostics completion on : N/A
SAS Address(0): 0x5003048000ee8a53
Connected Port Number: 1(path0)
Inquiry Data: MJ1311YNG6YYXAHitachi HDS5C3030ALA630 MEAOA5C0
FDE Enable: Disable
Secured: Unsecured
Locked: Unlocked
Needs EKM Attention: No
Foreign State: None
Device Speed: 6.0Gb/s
Link Speed: 6.0Gb/s
Media Type: Hard Disk Device
Drive Temperature :30C (86.00 F)
PI Eligibility: No
Drive is formatted for PI information: No
PI: No PI
Drive's write cache : Disabled
Drive's NCQ setting : Enabled
Port-0 :
Port status: Active
Port's Linkspeed: 6.0Gb/s
Drive has flagged a S.M.A.R.T alert : No
Enclosure Device ID: 36
Slot Number: 11
Drive's postion: DiskGroup: 19, Span: 0, Arm: 1
Enclosure position: 0
Device Id: 19
WWN:
Sequence Number: 4
Media Error Count: 0
Other Error Count: 0
Predictive Failure Count: 0
Last Predictive Failure Event Seq Number: 0
PD Type: SATA
Raw Size: 2.728 TB [0x15d50a3b0 Sectors]
Non Coerced Size: 2.728 TB [0x15d40a3b0 Sectors]
Coerced Size: 2.728 TB [0x15d400000 Sectors]
Firmware state: Online, Spun Up
Is Commissioned Spare : NO
Device Firmware Level: A580
Shield Counter: 0
Successful diagnostics completion on : N/A
SAS Address(0): 0x5003048000ee8e53
Connected Port Number: 0(path0)
Inquiry Data: MJ1313YNG1VA5CHitachi HDS5C3030ALA630 MEAOA580
FDE Enable: Disable
Secured: Unsecured
Locked: Unlocked
Needs EKM Attention: No
Foreign State: None
Device Speed: 6.0Gb/s
Link Speed: 6.0Gb/s
Media Type: Hard Disk Device
Drive Temperature :30C (86.00 F)
PI Eligibility: No
Drive is formatted for PI information: No
PI: No PI
Drive's write cache : Disabled
Drive's NCQ setting : Enabled
Port-0 :
Port status: Active
Port's Linkspeed: 6.0Gb/s
Drive has flagged a S.M.A.R.T alert : No
Atualização 28/06/12:
Eu finalmente tenho algumas novas informações sobre (o que achamos) a causa deste problema, então eu pensei em compartilhar.
Depois de entrar em contato com uma tecnologia Supermicro bem informada, eles nos forneceram uma ferramenta chamada Xflash (não parece estar prontamente disponível em seu FTP). Quando reunimos algumas informações usando esse utilitário, meu colega encontrou algo muito estranho:
root @ teste mogile2] # ./xflash.dat -i obter avail
Initializing Interface.
Expander: SAS2X36 (SAS2x36)
1) SAS2X36 (SAS2x36) (50030480:00EE917F) (0.0.0.0)
2) SAS2X36 (SAS2x36) (50030480:00E9D67F) (0.0.0.0)
3) SAS2X36 (SAS2x36) (50030480:0112D97F) (0.0.0.0)
Isso lista os gabinetes conectados. Você vê os 3 conectados (nós adicionamos uma terceira e uma quarta que ainda não estão aparecendo) com seus respectivos endereços SAS / WWN (50030480: 00EE917F). Agora podemos usar esse endereço para obter informações sobre os gabinetes individuais:
[root@mogile2 test]# ./xflash.dat -i 5003048000EE917F get exp
Initializing Interface.
Expander: SAS2X36 (SAS2x36)
Reading the expander information..........
Expander: SAS2X36 (SAS2x36) B3
SAS Address: 50030480:00EE917F
Enclosure Logical Id: 50030480:0000007F
IP Address: 0.0.0.0
Component Identifier: 0x0223
Component Revision: 0x05
[root@mogile2 test]# ./xflash.dat -i 5003048000E9D67F get exp
Initializing Interface.
Expander: SAS2X36 (SAS2x36)
Reading the expander information..........
Expander: SAS2X36 (SAS2x36) B3
SAS Address: 50030480:00E9D67F
Enclosure Logical Id: 50030480:0000007F
IP Address: 0.0.0.0
Component Identifier: 0x0223
Component Revision: 0x05
[root@mogile2 test]# ./xflash.dat -i 500304800112D97F get exp
Initializing Interface.
Expander: SAS2X36 (SAS2x36)
Reading the expander information..........
Expander: SAS2X36 (SAS2x36) B3
SAS Address: 50030480:0112D97F
Enclosure Logical Id: 50030480:0112D97F
IP Address: 0.0.0.0
Component Identifier: 0x0223
Component Revision: 0x05
Você pegou? O primeiro ID lógico de dois gabinetes é parcialmente mascarado onde o terceiro (que possui um ID de gabinete exclusivo correto) não é. Nós apontamos isso para a Supermicro e fomos capazes de confirmar que esse endereço deve ser definido durante a fabricação e houve um problema com um determinado lote desses gabinetes onde o ID lógico não foi definido.
Acreditamos que o controlador RAID está determinando o ID com base no ID lógico e, como nossos primeiros dois gabinetes possuem o mesmo ID lógico, eles obtêm o mesmo ID de gabinete. Também confirmamos que 0000007F é o padrão que vem do LSI como ID.
O próximo ponteiro que ajuda a confirmar isso poderia ser um problema de fabricação com uma execução de JBODs é o fato de que todos os 6 dos gabinetes que possuem esse problema começam com 00E . Acredito que entre 00E8 e 00EE Supermicro se esqueceu de programar os IDs lógicos corretamente e deixou de relembrar ou consertar o problema pós-produção.
Felizmente, para nós, existe uma ferramenta para gerenciar o WWN e o ID lógico dos dispositivos da Supermicro: ftp: // ftp .supermicro.com / utility / ExpanderXtools_Lite / . Nosso próximo passo é agendar um desligamento desses JBODs (após a migração de dados) e reprogramar o ID lógico e ver se ele resolve o problema.
Atualização 28/06/12 # 2:
Acabei de descobrir esta FAQ na Supermicro enquanto o Google pesquisava "lsi 0000007f": link . Eu ainda não entendo por que, nas últimas vezes em que entramos em contato com a Supermicro, eles nunca nos teriam direcionado para este artigo: \