Unable to list LUNs and to detach once attached
pgramenzi opened this issue · 18 comments
Initiator: iSCSIInitiator-1.0.0-beta5 on a MacBook Pro retina w/ OsX Yosemite (v10.10.5).
Target: Pure Storage FlashArray //m20
Symptom: I can successfully login to the target but no LUNs are listed and no way to logout from target.
I have configured a static target as follows:
Pieros-MacBook-Pro-2:~ piero$ iscsictl add target iqn.2010-06.com.purestorage:flasharray.168227283e2cc353,172.31.11.100
The specified target has been added
Pieros-MacBook-Pro-2:~ piero$ iscsictl list targets
iqn.2010-06.com.purestorage:flasharray.168227283e2cc353 <inactive, static>
172.31.11.100 <inactive, port 3260, interface default>
I have also configured an initiator as follows:
Pieros-MacBook-Pro-2:~ piero$ iscsictl list initiator-config
iqn.2015-01.com.localhost:initiator
node-alias localhost
Authentication: none
CHAP-name localhost
CHAP-secret
However, each time I try to login to that target I'm always getting a successful login but no LUNs are listed:
Pieros-MacBook-Pro-2:~ piero$ iscsictl login iqn.2010-06.com.purestorage:flasharray.168227283e2cc353
Attached
Also the “list luns” returns no existing LUNs from the active target:
Pieros-MacBook-Pro-2:iscsictl list luns
No active targets with LUNs exist
Needless to say that a 100GB test LUN was created on the array and associated to the host.
This is the output of tcpdump during the login:
No. Time Source Destination Protocol Length Info
6 0.000428 172.31.11.1 172.31.11.100 iSCSI 282 Login Command
Frame 6: 282 bytes on wire (2256 bits), 282 bytes captured (2256 bits)
Ethernet II, Src: Apple_cd:cf:c2 (0c:4d:e9💿cf:c2), Dst: PureStor_00:3f:f0 (24:a9:37:00:3f:f0)
Internet Protocol Version 4, Src: 172.31.11.1, Dst: 172.31.11.100
Transmission Control Protocol, Src Port: 49333, Dst Port: 3260, Seq: 1, Ack: 1, Len: 216
iSCSI (Login Command)
Opcode: Login Command (0x03)
1... .... = T: Transit to next login stage
.0.. .... = C: Text is complete
.... 00.. = CSG: Security negotiation (0x0)
.... ..01 = NSG: Operational negotiation (0x1)
VersionMax: 0x00
VersionMin: 0x00
TotalAHSLength: 0x00
DataSegmentLength: 165 (0x000000a5)
ISID: 800000000000
TSIH: 0x0000
InitiatorTaskTag: 0x00000000
CID: 0x0000
CmdSN: 0x00000000
ExpStatSN: 0x00000000
Key/Value Pairs
KeyValue: SessionType=Normal
KeyValue: InitiatorAlias=default
KeyValue: InitiatorName=iqn.2015-01.com.localhost
KeyValue: TargetName=iqn.2010-06.com.purestorage:flasharray.168227283e2cc353
KeyValue: AuthMethod=None
Padding: 000000
No. Time Source Destination Protocol Length Info
8 0.000684 172.31.11.100 172.31.11.1 iSCSI 182 Login Response (Success)
Frame 8: 182 bytes on wire (1456 bits), 182 bytes captured (1456 bits)
Ethernet II, Src: PureStor_00:3f:f0 (24:a9:37:00:3f:f0), Dst: Apple_cd:cf:c2 (0c:4d:e9💿cf:c2)
Internet Protocol Version 4, Src: 172.31.11.100, Dst: 172.31.11.1
Transmission Control Protocol, Src Port: 3260, Dst Port: 49333, Seq: 1, Ack: 217, Len: 116
iSCSI (Login Response)
Opcode: Login Response (0x23)
1... .... = T: Transit to next login stage
.0.. .... = C: Text is complete
.... 00.. = CSG: Security negotiation (0x0)
.... ..01 = NSG: Operational negotiation (0x1)
VersionMax: 0x00
VersionActive: 0x00
TotalAHSLength: 0x00
DataSegmentLength: 67 (0x00000043)
ISID: 800000000000
TSIH: 0x0000
InitiatorTaskTag: 0x00000000
StatSN: 0x2992e4c7
ExpCmdSN: 0x00000000
MaxCmdSN: 0x00000000
Status: Success (0x0000)
Key/Value Pairs
KeyValue: AuthMethod=None
KeyValue: TargetAlias=Pure FlashArray
KeyValue: TargetPortalGroupTag=1
Padding: 00
No. Time Source Destination Protocol Length Info
10 0.000877 172.31.11.1 172.31.11.100 iSCSI 398 Login Command
Frame 10: 398 bytes on wire (3184 bits), 398 bytes captured (3184 bits)
Ethernet II, Src: Apple_cd:cf:c2 (0c:4d:e9💿cf:c2), Dst: PureStor_00:3f:f0 (24:a9:37:00:3f:f0)
Internet Protocol Version 4, Src: 172.31.11.1, Dst: 172.31.11.100
Transmission Control Protocol, Src Port: 49333, Dst Port: 3260, Seq: 217, Ack: 117, Len: 332
iSCSI (Login Command)
Opcode: Login Command (0x03)
1... .... = T: Transit to next login stage
.0.. .... = C: Text is complete
.... 01.. = CSG: Operational negotiation (0x1)
.... ..11 = NSG: Full feature phase (0x3)
VersionMax: 0x00
VersionMin: 0x00
TotalAHSLength: 0x00
DataSegmentLength: 284 (0x0000011c)
ISID: 800000000000
TSIH: 0x0000
InitiatorTaskTag: 0x00000000
CID: 0x0000
CmdSN: 0x00000000
ExpStatSN: 0x2992e4c8
Key/Value Pairs
KeyValue: HeaderDigest=None
KeyValue: DefaultTime2Wait=2
KeyValue: ErrorRecoveryLevel=0
KeyValue: DataPDUInOrder=Yes
KeyValue: DataDigest=None
KeyValue: ImmediateData=Yes
KeyValue: MaxConnections=1
KeyValue: InitialR2T=Yes
KeyValue: MaxBurstLength=262144
KeyValue: DefaultTime2Retain=20
KeyValue: FirstBurstLength=65536
KeyValue: DataSequenceInOrder=Yes
KeyValue: MaxOutstandingR2T=1
KeyValue: MaxRecvDataSegmentLength=8192
No. Time Source Destination Protocol Length Info
11 0.001032 172.31.11.100 172.31.11.1 iSCSI 342 Login Response (Success)
Frame 11: 342 bytes on wire (2736 bits), 342 bytes captured (2736 bits)
Ethernet II, Src: PureStor_00:3f:f0 (24:a9:37:00:3f:f0), Dst: Apple_cd:cf:c2 (0c:4d:e9💿cf:c2)
Internet Protocol Version 4, Src: 172.31.11.100, Dst: 172.31.11.1
Transmission Control Protocol, Src Port: 3260, Dst Port: 49333, Seq: 117, Ack: 549, Len: 276
iSCSI (Login Response)
Opcode: Login Response (0x23)
1... .... = T: Transit to next login stage
.0.. .... = C: Text is complete
.... 01.. = CSG: Operational negotiation (0x1)
.... ..11 = NSG: Full feature phase (0x3)
VersionMax: 0x00
VersionActive: 0x00
TotalAHSLength: 0x00
DataSegmentLength: 228 (0x000000e4)
ISID: 800000000000
TSIH: 0x0004
InitiatorTaskTag: 0x00000000
StatSN: 0x2992e4c8
ExpCmdSN: 0x00000000
MaxCmdSN: 0x00000000
Status: Success (0x0000)
Key/Value Pairs
KeyValue: HeaderDigest=None
KeyValue: DataDigest=None
KeyValue: MaxConnections=1
KeyValue: ImmediateData=Yes
KeyValue: MaxRecvDataSegmentLength=262144
KeyValue: MaxBurstLength=262144
KeyValue: FirstBurstLength=65536
KeyValue: DefaultTime2Wait=2
KeyValue: DefaultTime2Retain=20
KeyValue: MaxOutstandingR2T=1
KeyValue: ErrorRecoveryLevel=0
No. Time Source Destination Protocol Length Info
13 0.001396 172.31.11.1 172.31.11.100 iSCSI 114 SCSI: Test Unit Ready LUN: 0x00
Frame 13: 114 bytes on wire (912 bits), 114 bytes captured (912 bits)
Ethernet II, Src: Apple_cd:cf:c2 (0c:4d:e9💿cf:c2), Dst: PureStor_00:3f:f0 (24:a9:37:00:3f:f0)
Internet Protocol Version 4, Src: 172.31.11.1, Dst: 172.31.11.100
Transmission Control Protocol, Src Port: 49333, Dst Port: 3260, Seq: 549, Ack: 393, Len: 48
iSCSI (SCSI Command)
Opcode: SCSI Command (0x01)
.0.. .... = I: Queued delivery
TotalAHSLength: 0x00
DataSegmentLength: 0 (0x00000000)
LUN
00.. .... = Address Mode: Simple logical unit addressing (0x00)
..00 0000 0000 0000 = LUN: 0x0000
InitiatorTaskTag: 0x00000000
ExpectedDataTransferLength: 0x00000000
CmdSN: 0x00000000
ExpStatSN: 0x2992e4c9
Response in: 14
Flags: 0x81, F, Attr: Simple
SCSI CDB Test Unit Ready
[LUN: 0x0000]
[Command Set:Direct Access Device (0x00) (Using default commandset)]
[Response in: 14]
Opcode: Test Unit Ready (0x00)
Control: 0x00
00.. .... = Vendor specific: 0x0
..00 0... = Reserved: 0x0
.... .0.. = NACA: Normal ACA is not set
.... ..0. = Obsolete: 0x0
.... ...0 = Obsolete: 0x0
No. Time Source Destination Protocol Length Info
14 0.001548 172.31.11.100 172.31.11.1 iSCSI 214 SCSI Response (Check Condition) LUN:0x00
Frame 14: 214 bytes on wire (1712 bits), 214 bytes captured (1712 bits)
Ethernet II, Src: PureStor_00:3f:f0 (24:a9:37:00:3f:f0), Dst: Apple_cd:cf:c2 (0c:4d:e9💿cf:c2)
Internet Protocol Version 4, Src: 172.31.11.100, Dst: 172.31.11.1
Transmission Control Protocol, Src Port: 3260, Dst Port: 49333, Seq: 393, Ack: 597, Len: 148
iSCSI (SCSI Response)
Opcode: SCSI Response (0x21)
Response: Command completed at target (0x00)
Status: Check Condition (0x02)
TotalAHSLength: 0x00
DataSegmentLength: 98 (0x00000062)
InitiatorTaskTag: 0x00000000
StatSN: 0x2992e4c9
ExpCmdSN: 0x00000001
MaxCmdSN: 0x00000080
ExpDataSN: 0x00000000
BidiReadResidualCount: 0x00000000
ResidualCount: 0x00000000
Request in: 13
Time from request: 0.000152000 seconds
SenseLength: 0x0060
Flags: 0x80
SCSI: SNS Info
[LUN: 0x0000]
.111 0000 = SNS Error Type: Current Error (0x70)
Valid: 112
0... .... = Filemark: False
.0.. .... = EOM: False
..0. .... = ILI: False
.... 0101 = Sense Key: Illegal Request (0x5)
Sense Info: 0x00000000
Additional Sense Length: 10
Command-Specific Information: 00000000
Additional Sense Code+Qualifier: Logical Unit Not Supported (0x2500)
Field Replaceable Unit Code: 0x00
0... .... = SKSV: False
.000 0000 0000 0000 0000 0000 = Sense Key Specific: 0x000000
and this is the dtruss of the iscsid daemon:
Pieros-MacBook-Pro-2:~ piero$ ps -ef | grep -i iscsi | grep -v grep
0 244 1 0 12:36PM ?? 0:00.02 /usr/local/libexec/iscsid
Pieros-MacBook-Pro-2:~ piero$ sudo dtruss -p 244
Password:
SYSCALL(args) = return
accept(0x4, 0x10BFDBBB0, 0x10BFDBBAC) = 7 0
sendto(0x5, 0x10BFE3EC3, 0x1) = 1 0
select(0x20, 0x7FFEF3D118C0, 0x7FFEF3D12950, 0x0, 0x0) = 1 0
recvfrom(0x6, 0x10BFDBA90, 0x100) = 1 0
close(0x8) = 0 0
setsockopt(0x7, 0xFFFF, 0x1005) = 0 0
setsockopt(0x7, 0xFFFF, 0x1006) = 0 0
recvfrom(0x7, 0x7FFF53E57E98, 0x18) = 24 0
recvmsg(0x7, 0x7FFF53E53DB0, 0x0) = 134 0
sendmsg(0x7, 0x7FFF53E53DB0, 0x0) = 24 0
recvfrom(0x7, 0x7FFF53E57E98, 0x18) = 24 0
recvfrom(0x7, 0x7FFF53E57F58, 0x18) = 24 0
recvmsg(0x7, 0x7FFF53E53E20, 0x0) = 166 0
workq_kernreturn(0x20, 0x0, 0x1) = 0 0
thread_selfid(0x0, 0x0, 0x0) = 8978 0
kevent64(0x3, 0x10BEDBDF8, 0x1) = 1 0
bsdthread_ctl(0x400, 0x1103, 0x800) = -1 Err#1
workq_kernreturn(0x20, 0x0, 0x1) = 0 0
kevent64(0x3, 0x0, 0x0) = 1 0
workq_kernreturn(0x20, 0x0, 0x1) = 0 0
workq_kernreturn(0x20, 0x0, 0x1) = 0 0
thread_selfid(0x0, 0x0, 0x0) = 8979 0
workq_kernreturn(0x20, 0x0, 0x1) = 0 0
kevent64(0x3, 0x10BF5EDF8, 0x1) = 1 0
kevent64(0x3, 0x7FFF787DD730, 0x1) = 1 0
bsdthread_ctl(0x400, 0x1103, 0x800) = -1 Err#1
workq_kernreturn(0x20, 0x0, 0x1) = 0 0
kevent64(0x3, 0x0, 0x0) = 1 0
workq_kernreturn(0x20, 0x0, 0x1) = 0 0
workq_kernreturn(0x20, 0x0, 0x1) = 0 0
workq_kernreturn(0x20, 0x0, 0x1) = 0 0
kevent64(0x3, 0x10BEDBDF8, 0x1) = 1 0
kevent64(0x3, 0x7FFF787DD730, 0x1) = 1 0
bsdthread_ctl(0x400, 0x1103, 0x800) = -1 Err#1
workq_kernreturn(0x20, 0x0, 0x1) = 0 0
kevent64(0x3, 0x0, 0x0) = 1 0
workq_kernreturn(0x20, 0x0, 0x1) = 0 0
workq_kernreturn(0x20, 0x0, 0x1) = 0 0
geteuid(0x20, 0x0, 0x1) = 0 0
geteuid(0x20, 0x0, 0x1) = 0 0
geteuid(0x20, 0x0, 0x1) = 0 0
geteuid(0x20, 0x0, 0x1) = 0 0
geteuid(0x20, 0x0, 0x1) = 0 0
sendto(0x7, 0x7FFF53E57EF0, 0x18) = 24 0
recvfrom(0x7, 0x7FFF53E57F58, 0x18) = 0 0
__disable_threadsignal(0x1, 0x0, 0x0) = 0 0
__disable_threadsignal(0x1, 0x0, 0x0) = 0 0
Once attached, the target won’t detach anymore and only a reboot helps:
Pieros-MacBook-Pro-2:~ piero$ iscsictl logout iqn.2010-06.com.purestorage:flasharray.168227283e2cc353
iscsictl: Input/output error
Pieros-MacBook-Pro-2:~ piero$ iscsictl list targets
iqn.2010-06.com.purestorage:flasharray.168227283e2cc353 <active, static, sid 0, tpgt 1, tsid 0x4>
172.31.11.100 <active, cid 0, port 3260, interface default>
This is the output of the tcpdump captured during the logout:
No. Time Source Destination Protocol Length Info
1 0.000000 172.31.11.100 172.31.11.1 iSCSI 114 NOP In
Frame 1: 114 bytes on wire (912 bits), 114 bytes captured (912 bits)
Ethernet II, Src: PureStor_00:3f:f0 (24:a9:37:00:3f:f0), Dst: Apple_cd:cf:c2 (0c:4d:e9💿cf:c2)
Internet Protocol Version 4, Src: 172.31.11.100, Dst: 172.31.11.1
Transmission Control Protocol, Src Port: 3260, Dst Port: 49333, Seq: 1, Ack: 1, Len: 48
iSCSI (NOP In)
Opcode: NOP In (0x20)
TotalAHSLength: 0x00
DataSegmentLength: 0 (0x00000000)
LUN
00.. .... = Address Mode: Simple logical unit addressing (0x00)
..00 0000 0000 0000 = LUN: 0x0000
InitiatorTaskTag: 0xffffffff
TargetTransferTag: 0x00000017
StatSN: 0x2992e4ca
ExpCmdSN: 0x00000001
MaxCmdSN: 0x00000080
No. Time Source Destination Protocol Length Info
3 0.000081 172.31.11.1 172.31.11.100 iSCSI 114 NOP Out
Frame 3: 114 bytes on wire (912 bits), 114 bytes captured (912 bits)
Ethernet II, Src: Apple_cd:cf:c2 (0c:4d:e9💿cf:c2), Dst: PureStor_00:3f:f0 (24:a9:37:00:3f:f0)
Internet Protocol Version 4, Src: 172.31.11.1, Dst: 172.31.11.100
Transmission Control Protocol, Src Port: 49333, Dst Port: 3260, Seq: 1, Ack: 49, Len: 48
iSCSI (NOP Out)
Opcode: NOP Out (0x00)
.1.. .... = I: Immediate delivery
TotalAHSLength: 0x00
DataSegmentLength: 0 (0x00000000)
LUN
00.. .... = Address Mode: Simple logical unit addressing (0x00)
..00 0000 0000 0000 = LUN: 0x0000
InitiatorTaskTag: 0xffffffff
TargetTransferTag: 0x00000017
CmdSN: 0x00000001
ExpStatSN: 0x2992e4ca
No. Time Source Destination Protocol Length Info
8 11.877748 172.31.11.1 172.31.11.100 iSCSI 218 Login Command
Frame 8: 218 bytes on wire (1744 bits), 218 bytes captured (1744 bits)
Ethernet II, Src: Apple_cd:cf:c2 (0c:4d:e9💿cf:c2), Dst: PureStor_00:3f:f0 (24:a9:37:00:3f:f0)
Internet Protocol Version 4, Src: 172.31.11.1, Dst: 172.31.11.100
Transmission Control Protocol, Src Port: 49355, Dst Port: 3260, Seq: 1, Ack: 1, Len: 152
iSCSI (Login Command)
Opcode: Login Command (0x03)
1... .... = T: Transit to next login stage
.0.. .... = C: Text is complete
.... 00.. = CSG: Security negotiation (0x0)
.... ..01 = NSG: Operational negotiation (0x1)
VersionMax: 0x00
VersionMin: 0x00
TotalAHSLength: 0x00
DataSegmentLength: 101 (0x00000065)
ISID: 800000000001
TSIH: 0x0000
InitiatorTaskTag: 0x00000000
CID: 0x0000
CmdSN: 0x00000000
ExpStatSN: 0x00000000
Key/Value Pairs
KeyValue: SessionType=Discovery
KeyValue: InitiatorAlias=default
KeyValue: InitiatorName=iqn.2015-01.com.localhost
KeyValue: AuthMethod=None
Padding: 000000
No. Time Source Destination Protocol Length Info
10 11.877912 172.31.11.100 172.31.11.1 iSCSI 182 Login Response (Success)
Frame 10: 182 bytes on wire (1456 bits), 182 bytes captured (1456 bits)
Ethernet II, Src: PureStor_00:3f:f0 (24:a9:37:00:3f:f0), Dst: Apple_cd:cf:c2 (0c:4d:e9💿cf:c2)
Internet Protocol Version 4, Src: 172.31.11.100, Dst: 172.31.11.1
Transmission Control Protocol, Src Port: 3260, Dst Port: 49355, Seq: 1, Ack: 153, Len: 116
iSCSI (Login Response)
Opcode: Login Response (0x23)
1... .... = T: Transit to next login stage
.0.. .... = C: Text is complete
.... 00.. = CSG: Security negotiation (0x0)
.... ..01 = NSG: Operational negotiation (0x1)
VersionMax: 0x00
VersionActive: 0x00
TotalAHSLength: 0x00
DataSegmentLength: 67 (0x00000043)
ISID: 800000000001
TSIH: 0x0000
InitiatorTaskTag: 0x00000000
StatSN: 0x02897e86
ExpCmdSN: 0x00000000
MaxCmdSN: 0x00000000
Status: Success (0x0000)
Key/Value Pairs
KeyValue: AuthMethod=None
KeyValue: TargetAlias=Pure FlashArray
KeyValue: TargetPortalGroupTag=1
Padding: 00
No. Time Source Destination Protocol Length Info
12 11.878041 172.31.11.1 172.31.11.100 iSCSI 398 Login Command
Frame 12: 398 bytes on wire (3184 bits), 398 bytes captured (3184 bits)
Ethernet II, Src: Apple_cd:cf:c2 (0c:4d:e9💿cf:c2), Dst: PureStor_00:3f:f0 (24:a9:37:00:3f:f0)
Internet Protocol Version 4, Src: 172.31.11.1, Dst: 172.31.11.100
Transmission Control Protocol, Src Port: 49355, Dst Port: 3260, Seq: 153, Ack: 117, Len: 332
iSCSI (Login Command)
Opcode: Login Command (0x03)
1... .... = T: Transit to next login stage
.0.. .... = C: Text is complete
.... 01.. = CSG: Operational negotiation (0x1)
.... ..11 = NSG: Full feature phase (0x3)
VersionMax: 0x00
VersionMin: 0x00
TotalAHSLength: 0x00
DataSegmentLength: 284 (0x0000011c)
ISID: 800000000001
TSIH: 0x0000
InitiatorTaskTag: 0x00000000
CID: 0x0000
CmdSN: 0x00000000
ExpStatSN: 0x02897e87
Key/Value Pairs
KeyValue: HeaderDigest=None
KeyValue: DefaultTime2Wait=2
KeyValue: ErrorRecoveryLevel=0
KeyValue: DataPDUInOrder=Yes
KeyValue: DataDigest=None
KeyValue: ImmediateData=Yes
KeyValue: MaxConnections=1
KeyValue: InitialR2T=Yes
KeyValue: MaxBurstLength=262144
KeyValue: DefaultTime2Retain=20
KeyValue: FirstBurstLength=65536
KeyValue: DataSequenceInOrder=Yes
KeyValue: MaxOutstandingR2T=1
KeyValue: MaxRecvDataSegmentLength=8192
No. Time Source Destination Protocol Length Info
13 11.878161 172.31.11.100 172.31.11.1 iSCSI 342 Login Response (Success)
Frame 13: 342 bytes on wire (2736 bits), 342 bytes captured (2736 bits)
Ethernet II, Src: PureStor_00:3f:f0 (24:a9:37:00:3f:f0), Dst: Apple_cd:cf:c2 (0c:4d:e9💿cf:c2)
Internet Protocol Version 4, Src: 172.31.11.100, Dst: 172.31.11.1
Transmission Control Protocol, Src Port: 3260, Dst Port: 49355, Seq: 117, Ack: 485, Len: 276
iSCSI (Login Response)
Opcode: Login Response (0x23)
1... .... = T: Transit to next login stage
.0.. .... = C: Text is complete
.... 01.. = CSG: Operational negotiation (0x1)
.... ..11 = NSG: Full feature phase (0x3)
VersionMax: 0x00
VersionActive: 0x00
TotalAHSLength: 0x00
DataSegmentLength: 228 (0x000000e4)
ISID: 800000000001
TSIH: 0x001d
InitiatorTaskTag: 0x00000000
StatSN: 0x02897e87
ExpCmdSN: 0x00000000
MaxCmdSN: 0x00000000
Status: Success (0x0000)
Key/Value Pairs
KeyValue: HeaderDigest=None
KeyValue: DataDigest=None
KeyValue: MaxConnections=1
KeyValue: ImmediateData=Yes
KeyValue: MaxRecvDataSegmentLength=262144
KeyValue: MaxBurstLength=262144
KeyValue: FirstBurstLength=65536
KeyValue: DefaultTime2Wait=2
KeyValue: DefaultTime2Retain=20
KeyValue: MaxOutstandingR2T=1
KeyValue: ErrorRecoveryLevel=0
No. Time Source Destination Protocol Length Info
15 11.878333 172.31.11.1 172.31.11.100 iSCSI 130 Text Command
Frame 15: 130 bytes on wire (1040 bits), 130 bytes captured (1040 bits)
Ethernet II, Src: Apple_cd:cf:c2 (0c:4d:e9💿cf:c2), Dst: PureStor_00:3f:f0 (24:a9:37:00:3f:f0)
Internet Protocol Version 4, Src: 172.31.11.1, Dst: 172.31.11.100
Transmission Control Protocol, Src Port: 49355, Dst Port: 3260, Seq: 485, Ack: 393, Len: 64
iSCSI (Text Command)
Opcode: Text Command (0x04)
.1.. .... = I: Immediate delivery
Flags: 0x80
TotalAHSLength: 0x00
DataSegmentLength: 16 (0x00000010)
LUN
00.. .... = Address Mode: Simple logical unit addressing (0x00)
..00 0000 0000 0000 = LUN: 0x0000
InitiatorTaskTag: 0x00000000
TargetTransferTag: 0xffffffff
CmdSN: 0x00000000
ExpStatSN: 0x02897e88
Key/Value Pairs
KeyValue: SendTargets=All
No. Time Source Destination Protocol Length Info
16 11.878458 172.31.11.100 172.31.11.1 iSCSI 382 Text Response
Frame 16: 382 bytes on wire (3056 bits), 382 bytes captured (3056 bits)
Ethernet II, Src: PureStor_00:3f:f0 (24:a9:37:00:3f:f0), Dst: Apple_cd:cf:c2 (0c:4d:e9💿cf:c2)
Internet Protocol Version 4, Src: 172.31.11.100, Dst: 172.31.11.1
Transmission Control Protocol, Src Port: 3260, Dst Port: 49355, Seq: 393, Ack: 549, Len: 316
iSCSI (Text Response)
Opcode: Text Response (0x24)
Flags: 0x80
TotalAHSLength: 0x00
DataSegmentLength: 267 (0x0000010b)
LUN
00.. .... = Address Mode: Simple logical unit addressing (0x00)
..00 0000 0000 0000 = LUN: 0x0000
InitiatorTaskTag: 0x00000000
TargetTransferTag: 0xffffffff
StatSN: 0x02897e88
ExpCmdSN: 0x00000000
MaxCmdSN: 0x00000000
Key/Value Pairs
KeyValue: TargetName=iqn.2010-06.com.purestorage:flasharray.168227283e2cc353
KeyValue: TargetAddress=10.0.24.98:3260,1
KeyValue: TargetAddress=10.0.24.108:3260,1
KeyValue: TargetAddress=10.0.24.99:3260,1
KeyValue: TargetAddress=10.0.24.109:3260,1
KeyValue: TargetAddress=172.31.11.100:3260,1
KeyValue: TargetAddress=172.31.11.101:3260,1
Padding: 00
No. Time Source Destination Protocol Length Info
19 11.878763 172.31.11.100 172.31.11.1 iSCSI 114 Logout Response
Frame 19: 114 bytes on wire (912 bits), 114 bytes captured (912 bits)
Ethernet II, Src: PureStor_00:3f:f0 (24:a9:37:00:3f:f0), Dst: Apple_cd:cf:c2 (0c:4d:e9💿cf:c2)
Internet Protocol Version 4, Src: 172.31.11.100, Dst: 172.31.11.1
Transmission Control Protocol, Src Port: 3260, Dst Port: 49355, Seq: 709, Ack: 597, Len: 48
iSCSI (Logout Response)
Opcode: Logout Response (0x26)
Response: Connection closed successfully (0x00)
TotalAHSLength: 0x00
DataSegmentLength: 0 (0x00000000)
InitiatorTaskTag: 0x00000000
StatSN: 0x02897e89
ExpCmdSN: 0x00000000
MaxCmdSN: 0x00000000
Time2Wait: 0x0000
Time2Retain: 0x0000
again, the dtruss capture of the logout session is as follows:
Pieros-MacBook-Pro-2:~ piero$ sudo dtruss -p 244
Password:
SYSCALL(args) = return
accept(0x4, 0x10BFDBBB0, 0x10BFDBBAC) = 8 0
sendto(0x5, 0x10BFE3EC3, 0x1) = 1 0
select(0x20, 0x7FFEF3D118C0, 0x7FFEF3D12950, 0x0, 0x0) = 1 0
recvfrom(0x6, 0x10BFDBA90, 0x100) = 1 0
close(0x7) = 0 0
setsockopt(0x8, 0xFFFF, 0x1005) = 0 0
setsockopt(0x8, 0xFFFF, 0x1006) = 0 0
recvfrom(0x8, 0x7FFF53E57E98, 0x18) = 24 0
recvmsg(0x8, 0x7FFF53E53DB0, 0x0) = 134 0
sendto(0x8, 0x7FFF53E57E48, 0x18) = 24 0
recvfrom(0x8, 0x7FFF53E57E98, 0x18) = 24 0
recvfrom(0x8, 0x7FFF53E57F58, 0x18) = 24 0
recvmsg(0x8, 0x7FFF53E53E00, 0x0) = 166 0
workq_kernreturn(0x20, 0x0, 0x1) = 0 0
thread_selfid(0x0, 0x0, 0x0) = 11860 0
kevent64(0x3, 0x10BEDBDF8, 0x1) = 1 0
bsdthread_ctl(0x400, 0x1103, 0x800) = -1 Err#1
workq_kernreturn(0x20, 0x0, 0x1) = 0 0
kevent64(0x3, 0x0, 0x0) = 1 0
workq_kernreturn(0x20, 0x0, 0x1) = 0 0
workq_kernreturn(0x20, 0x0, 0x1) = 0 0
thread_selfid(0x0, 0x0, 0x0) = 11861 0
workq_kernreturn(0x20, 0x0, 0x1) = 0 0
kevent64(0x3, 0x10BF5EDF8, 0x1) = 1 0
kevent64(0x3, 0x7FFF787DD730, 0x1) = 1 0
bsdthread_ctl(0x400, 0x1103, 0x800) = -1 Err#1
workq_kernreturn(0x20, 0x0, 0x1) = 0 0
kevent64(0x3, 0x0, 0x0) = 1 0
workq_kernreturn(0x20, 0x0, 0x1) = 0 0
workq_kernreturn(0x20, 0x0, 0x1) = 0 0
workq_kernreturn(0x20, 0x0, 0x1) = 0 0
kevent64(0x3, 0x10BF5EDF8, 0x1) = 1 0
kevent64(0x3, 0x7FFF787DD730, 0x1) = 1 0
bsdthread_ctl(0x400, 0x1103, 0x800) = -1 Err#1
workq_kernreturn(0x20, 0x0, 0x1) = 0 0
kevent64(0x3, 0x0, 0x0) = 1 0
workq_kernreturn(0x20, 0x0, 0x1) = 0 0
workq_kernreturn(0x20, 0x0, 0x1) = 0 0
recvfrom(0x8, 0x7FFF53E57F58, 0x18) = -1 Err#35
__disable_threadsignal(0x1, 0x0, 0x0) = 0 0
__disable_threadsignal(0x1, 0x0, 0x0) = 0 0
thread_selfid(0x10BEDE000, 0x83000, 0x1F37) = 11894 0
thread_selfid(0x10BE5B000, 0x83000, 0x1B37) = 11897 0
bsdthread_create(0x10BDB02A9, 0x0, 0x80000) = 200130560 0
thread_selfid(0x0, 0x0, 0x0) = 11921 0
__disable_threadsignal(0x1, 0x0, 0x0) = 0 0
thread_selfid(0x10BE5B000, 0x83000, 0x1F3B) = 11923 0
Thanks in advance for your help,
Piero
Hi Piero, can you please capture all of the traffic using Wireshark or tcpdump into a file?
What you've posted is partial and I don't see issues, plus it's very difficult to follow in this format.
If you just let the machine sit for about a minute or so, does automatically drop the target on its own (i.e., list targets would show inactive)?
@pgramenzi I think the attachment is missing...please upload again.
Attached a ZIP archive with iscsi-login.snoop and iscsi-logout.snoop.
Just to answer to your second question, no, the target does not fall back into inactive state:
Pieros-MacBook-Pro-2:~ piero$ date
Mon Mar 13 15:28:20 CET 2017
Pieros-MacBook-Pro-2:~ piero$ iscsictl list targets
iqn.2010-06.com.purestorage:flasharray.168227283e2cc353 <active, static, sid 0, tpgt 1, tsid 0x1>
172.31.11.100 <active, cid 0, port 3260, interface default>
Pieros-MacBook-Pro-2:~ piero$ date
Mon Mar 13 15:34:19 CET 2017
Pieros-MacBook-Pro-2:~ piero$ iscsictl list targets
iqn.2010-06.com.purestorage:flasharray.168227283e2cc353 <active, static, sid 0, tpgt 1, tsid 0x1>
172.31.11.100 <active, cid 0, port 3260, interface default>
Thanks. I don't see anything strange in these logs, except that the target returns a "Check Condition" response to a Test Unit Ready command. Specifically, the sense data says that "Logical Unit Not Supported". Basically the target is rejecting a command that is normally issued, so the OS doesn't do anything from there on.
Are you positive that your setup is working correctly? Can you verify using globalSAN or another initiator?
Two more things: (1) What does the macOS disk utility show once you login/attach? (2) Can you start recording traffic, login, let it record for ~ 2 minutes, logout and send me the trace? I want to see what's going on in between. Thanks.
-
The disk utility doesn't show anything other than disks physically attached to the MacBook. See screen capture attached.
-
Please find the full tcpdump I captured as requested: started capturing, let it go for 30sec or so, login, let it go for 2min or so, logout, let it go for 30sec or so.
This is the sequence of commands I have issues from login to logout:
Pieros-MacBook-Pro-2:~ piero$ iscsictl login iqn.2010-06.com.purestorage:flasharray.168227283e2cc353
Attached Pieros-MacBook-Pro-2:iscsictl list targets
iqn.2010-06.com.purestorage:flasharray.168227283e2cc353 <active, static, sid 0, tpgt 1, tsid 0x1>
172.31.11.100 <active, cid 0, port 3260, interface default>
Pieros-MacBook-Pro-2:~ piero$ iscsictl list luns
No active targets with LUNs exist
Pieros-MacBook-Pro-2:~ piero$ iscsictl logout iqn.2010-06.com.purestorage:flasharray.168227283e2cc353
iscsictl: Input/output error
Best,
Piero
The error is a SCSI error, not an iSCSI error, which means the OS sends a request that the target doesn't like. The initiator doesn't control those requests, it simply passes them onto iSCSI and vice versa. So based on the logs, I can't determine that there is anything wrong with the iSCSI layer.
Is there an iSCSI target configuration that you can share with me?
Can you try globalSAN on the same Mac to ensure that it is not an macOS specific issue?
With globalSAN it works fine. I had to change the default iSCSI name that globalSAN had assigned to my Mac from naa.294100eb3e4447fc9eeeba9fccfab435 to iqn.naa.294100eb3e4447fc9eeeba9fccfab435, and enable the "Always send Session Type when connecting" flag from the iSCSI Option panel.
Attached you can find two tcpdumps, one captured with no flag set, and the other one with the flag set.
When the flag is set, the "connect" is causing the autodiscovery of the available LUNs and the automount. So the tcpdump contains the rescan, connect, discovery of available LUNs, mount, umount, disconnect.
It appears as though you have not configured the initiator name properly on iSCSI initiator - it is still set to the default, whereas with globalSAN you did configure it correctly. Is your NAS set to restrict access to specified initiators only? Can you set the initiator name and try again?
Understood. I just want to make sure your target isn't doing some weird blocking on a LUN basis. The only other thing that I can think of is that macOS doesn't like or does something weird when you don't have a LUN 0. It looks like your LUNs start at 1 is that correct?
As I mentioned earlier, this code uses the macOS SCSI stack. globalSAN for example may have its own SCSI stack or a modified version of the macOS stack I'm not sure. I basically just the let the OS do it's thing, and it gives up when the target responds for LUN 0 with "Illegal request" and "logical unit not supported".
If it is easy to change LUN Ids, I'd appreciate switching to 1 and seeing if that works. I'll see if I can "intercept" the request from the OS but I'm not sure.
Definitely an issue with the SCSI stack & its interaction with iSCSI. FYI, the globalSAN flag is entirely unrelated. The iSCSI standard says the "SessionType" key is optional, but some targets fail without it; so those targets are actually less compliant with the standard. I always send that key along, even though it is optional.
Now to the problem. Basically, when a LUN does not exist the iSCSI target sends back a CHECK_CONDITION along with some sense data. macOS, upon receiving a CHECK_CONDITION will follow up to get sense data. However, when sense data is received it expects the condition to be GOOD. The issue is that the CHECK_CONDITION is accompanied by sense data obviating the need for macOS to follow up, but the OS doesn't know that. So I've included a flag now that sets the task status to GOOD when sense data is sent along. I had to look at Apple's open source driver stack to determine all of this.
I've attached a new build for you to try out. I've reproduced the problem using a VM and this build now fixes the issue. Please confirm. Replace the .zip extension with .dmg and install as usual (it's a disk image, not a zip file). Thanks.
Hi Nareg,
this one worked like a charm and once the Mac went connected to the target, it was able to automatically discover the LUN and mount it with no additional actions from my side:
Pieros-MacBook-Pro-2:~ piero$ iscsictl list initiator-config
iqn.2015-01.com.localhost:initiator
node-alias localhost
Authentication: none
CHAP-name localhost
CHAP-secret
Pieros-MacBook-Pro-2:~ piero$ iscsictl list discovery-config
SendTargets: enabled
interval: 60 seconds
discovery-portals: 1
172.31.11.100 <port 3260, interface default>
Pieros-MacBook-Pro-2:~ piero$ iscsictl list targets
iqn.2010-06.com.purestorage:flasharray.168227283e2cc353 <inactive, static>
172.31.11.100 <inactive, port 3260, interface default>
Pieros-MacBook-Pro-2:~ piero$ iscsictl login iqn.2010-06.com.purestorage:flasharray.168227283e2cc353
Attached iqn.2010-06.com.purestorage:flasharray.168227283e2cc353 <scsi domain 1, target 0>
PURE FlashArray 100
Serial Number n/a
lun 1: type 0x00 (Block device)
lun 0: type 0x1f (Unknown or no device)
Pieros-MacBook-Pro-2:~ piero$ iscsictl list targets
iqn.2010-06.com.purestorage:flasharray.168227283e2cc353 <active, static, sid 0, tpgt 1, tsid 0x1>
172.31.11.100 <active, cid 0, port 3260, interface default>
Pieros-MacBook-Pro-2:~ piero$ iscsictl list luns
iqn.2010-06.com.purestorage:flasharray.168227283e2cc353 <scsi domain 1, target 0>
PURE FlashArray 100
Serial Number n/a
lun 1: type 0x00 (Block device)
disk3: 2,2 TB (4294967296 512 byte blocks)
lun 0: type 0x1f (Unknown or no device)
Pieros-MacBook-Pro-2:~ piero$ iscsictl logout iqn.2010-06.com.purestorage:flasharray.168227283e2cc353
Detached iqn.2010-06.com.purestorage:flasharray.168227283e2cc353
Pieros-MacBook-Pro-2:~ piero$ iscsictl list targets
iqn.2010-06.com.purestorage:flasharray.168227283e2cc353 <inactive, static>
172.31.11.100 <inactive, port 3260, interface default>
Thanks,
Piero
Great, thanks for testing. Keep using this build and the changes will be incorporated into the next release.