关于以太网PAUSE的一点研究

在学习自动协商时,发现有以太网PAUSE帧这个东西,很网上资料很少,有讲到的也是很简略,似乎这是一个神秘的不想让人知道的东西。这里将自己对PAUSE帧的一些研究,对于太术语的方面,直接参考网上资料,不在这里提及。

疑惑

在使用ethtool查看网卡信息中,有一个比较疑惑的地方,它就是“Advertised pause frame use:”,有的是”No”,有的是“ Symmetric”,不同芯片平台信息不同,在ARM上,一般是No,而在x86上,一般是Symmetric,以此信息搜索,网络资料少得可怜,甚至说没有,没法,只能跟踪代码。
下面就是ethtool的一个比较经典的输出:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
root@latelee:~# ethtool eth0
Settings for eth0:
Supported ports: [ TP AUI BNC MII FIBRE ]
Supported link modes: 10baseT/Half 10baseT/Full
100baseT/Half 100baseT/Full
1000baseT/Full
Supports auto-negotiation: Yes
Advertised link modes: 10baseT/Half 10baseT/Full
100baseT/Half 100baseT/Full
1000baseT/Full
Advertised pause frame use: Symmetric Receive-only
Advertised auto-negotiation: Yes
Speed: 1000Mb/s
Duplex: Full
Port: MII
PHYAD: 0
Transceiver: external
Auto-negotiation: on
Current message level: 0x00000000 (0)

Link detected: yes

可以看到,这回变成“Symmetric Receive-only”了。 首先找到ethtool源码,以前写有文章讲过,但那时使用的版本旧,我担心版本的问题,于是找到新的版本的官网:https://www.kernel.org/pub/software/network/ethtool/,网址是Linux内核官网上的。最新的版本是3.18。我对比旧的和新的版本,发现在打印“Advertised pause frame use:”所用的代码是相同的。如下:

1
2
3
4
5
6
7
8
9
10
11
12
fprintf(stdout, "     %s pause frame use: ", prefix);
if (mask & ADVERTISED_Pause) {
fprintf(stdout, "Symmetric");
if (mask & ADVERTISED_Asym_Pause)
fprintf(stdout, " Receive-only");
fprintf(stdout, "\n");
} else {
if (mask & ADVERTISED_Asym_Pause)
fprintf(stdout, "Transmit-only\n");
else
fprintf(stdout, "No\n");
}

其中mask是ethtool_cmd结构体的advertising成员,ADVERTISED_Pause和ADVERTISED_Asym_Pause定义如下:

1
2
#define ADVERTISED_Pause        (1 << 13)
#define ADVERTISED_Asym_Pause (1 << 14)

可以得到如下结论:

ADVERTISED_Pause ADVERTISED_Asym_Pause 结果
0 0 No
0 1 Symmetric Transmit-only
1 0 Symmetric
1 1 Symmetric Receive-only

至于ethtool为什么会如此打印,这方面资料也少,幸好,在无意中,看到802.3标志有关于PAUSE的描述。但还是有人对标准文档的描述有疑问的:

后来又查了下,发现关于此解释有新版本了,见附录。

标准

在跟踪内核代码之前,先了解802.3标准有关描述。在标准中,关于PAUSE主要有PAUSE和ASM_DIR两个标志,分别对应于PS1和PS2。如下图所示:

右侧说的4.7和4.8似乎说的是PHY的第4个寄存器,在看PHY芯片时,发现对应不上来,但寄存器4的确有相关的位说明,因此认为上图所说的和寄存器是对应的关系。下面是Intel一款PHY芯片手册的截图:

描述如下:

从描述上看到,PAUSE和ASM_DIR的不同取值,将影响PAUSE的表现。在这里,当PAUSE和ASM_DIR分别取0和1时,表示从本地设备到对端的异步暂停,亦即“Transmit-only”,至于最后一项,ethtool打印的和标准文档所说的有点出入,我暂时还没有研究透。在某些角度上说,好像是可以得到结论:PAUSE、ASM_DIR和ethtool(内核定义)使用的ADVERTISED_Pause和ADVERTISED_Asym_Pause是一一对应的关系。
关于本地设备(即网卡)和对端设备(如交换机)的详细关系如下:

驱动

在看以太网流控方面的资料时,发现3.17版本内核打印网络状态函数新增了flow control的信息,如下:

1
2
3
4
5
6
7
8
9
10
11
12
void phy_print_status(struct phy_device *phydev)
{
if (phydev->link) {
netdev_info(phydev->attached_dev,
"Link is Up - %s/%s - flow control %s\n",
phy_speed_to_str(phydev->speed),
DUPLEX_FULL == phydev->duplex ? "Full" : "Half",
phydev->pause ? "rx/tx" : "off");
} else {
netdev_info(phydev->attached_dev, "Link is Down\n");
}
}

只判断pause值为1即认为流控为rx/tx,否则为off,目前我还不知道为什么是这样判断。 而赋值相关的代码在genphy_read_status函数中,如下:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
lpa = phy_read(phydev, MII_LPA);
if (lpa < 0)
return lpa;

phydev->lp_advertising |= mii_lpa_to_ethtool_lpa_t(lpa);

adv = phy_read(phydev, MII_ADVERTISE);
if (adv < 0)
return adv;

common_adv = lpa & adv;

phydev->speed = SPEED_10;
phydev->duplex = DUPLEX_HALF;
phydev->pause = 0;
phydev->asym_pause = 0;

if (common_adv_gb & (LPA_1000FULL | LPA_1000HALF)) {
phydev->speed = SPEED_1000;

if (common_adv_gb & LPA_1000FULL)
phydev->duplex = DUPLEX_FULL;
} else if (common_adv & (LPA_100FULL | LPA_100HALF)) {
phydev->speed = SPEED_100;

if (common_adv & LPA_100FULL)
phydev->duplex = DUPLEX_FULL;
} else
if (common_adv & LPA_10FULL)
phydev->duplex = DUPLEX_FULL;

if (phydev->duplex == DUPLEX_FULL) {
phydev->pause = lpa & LPA_PAUSE_CAP ? 1 : 0;
phydev->asym_pause = lpa & LPA_PAUSE_ASYM ? 1 : 0;
}

根据第5个寄存器MII_LPA的值判断对端(如交换机)的情况,如果对端支持LPA_PAUSE_CAP、LPA_PAUSE_ASYM就分别赋值到pause、asym_pause。从上面的分析可以知道,这两个可以和寄存器4的值对应起来,也可以和标准文档说的PAUSE和ASM_DIR对应起来。
再回到前面ethtool代码出现的ADVERTISED_Pause和ADVERTISED_Asym_Pause。内核定义了许多ADVERTISED_XXX的宏,它表示接口通告的能力(这个有点难理解,我还不太应该怎么描述,感觉就是在自动协商时,将“能力”告诉对端,让对方知道你具备哪些能力),对应寄存器4(MII_ADVERTISE)。相应的,有SUPPORTED_XXX宏定义,它表示接口所支持的能力、特性。
PHY在自动协商时会调用到genphy_config_advert函数(注意!该函数不对PAUSE标志ADVERTISE_PAUSE_CAP进行特别处理,只是根据phydev->advertising来判断):

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
static int genphy_config_advert(struct phy_device *phydev)
{
u32 advertise;
int oldadv, adv, bmsr;
int err, changed = 0;

/* Only allow advertising what this PHY supports */
phydev->advertising &= phydev->supported; // 先获取本地网卡所支持的特性
advertise = phydev->advertising;

/* Setup standard advertisement */
adv = phy_read(phydev, MII_ADVERTISE); // 读取寄存器4的值
if (adv < 0)
return adv;

oldadv = adv;
adv &= ~(ADVERTISE_ALL | ADVERTISE_100BASE4 | ADVERTISE_PAUSE_CAP |
ADVERTISE_PAUSE_ASYM);
adv |= ethtool_adv_to_mii_adv_t(advertise); // 将内核支持特性的格式转换成PHY芯片识别的,因为adv要写到寄存器

if (adv != oldadv) {
err = phy_write(phydev, MII_ADVERTISE, adv);

if (err < 0)
return err;
changed = 1;
}

// ...
}

该函数调用了ethtool_adv_to_mii_adv_t,在这个函数里面判断特性,相关代码如下:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
static inline u32 ethtool_adv_to_mii_adv_t(u32 ethadv)
{
u32 result = 0;

if (ethadv & ADVERTISED_10baseT_Half)
result |= ADVERTISE_10HALF;
if (ethadv & ADVERTISED_10baseT_Full)
result |= ADVERTISE_10FULL;
if (ethadv & ADVERTISED_100baseT_Half)
result |= ADVERTISE_100HALF;
if (ethadv & ADVERTISED_100baseT_Full)
result |= ADVERTISE_100FULL;
if (ethadv & ADVERTISED_Pause)
result |= ADVERTISE_PAUSE_CAP;
if (ethadv & ADVERTISED_Asym_Pause)
result |= ADVERTISE_PAUSE_ASYM;

return result;
}

为了和文章开头对应,最后跟踪一下advertising值。
ethtool获取网卡信息:

1
phy_ethtool_gset -> cmd->advertising = phydev->advertising;

PHY驱动定义:

1
2
3
4
5
6
7
8
9
10
11
static struct phy_driver genphy_driver[] = 
{
.phy_id = 0xffffffff,
.phy_id_mask = 0xffffffff,
.name = "Generic PHY",
.soft_reset = genphy_soft_reset,
.config_init = genphy_config_init,
.features = PHY_GBIT_FEATURES | SUPPORTED_MII |
SUPPORTED_AUI | SUPPORTED_FIBRE |
SUPPORTED_BNC,
}

在phy_probe会赋值,但后面的函数会将其覆盖掉:

1
2
3
phy_probe
-> phydev->supported = phydrv->features;
-> phydev->advertising = phydev->supported;

在phy初始化时,会调用genphy_config_init:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
int genphy_config_init(struct phy_device *phydev)
{
int val;
u32 features;

features = (SUPPORTED_TP | SUPPORTED_MII
| SUPPORTED_AUI | SUPPORTED_FIBRE |
SUPPORTED_BNC);

/* Do we support autonegotiation? */
val = phy_read(phydev, MII_BMSR);
if (val < 0)
return val;

if (val & BMSR_ANEGCAPABLE)
features |= SUPPORTED_Autoneg;

if (val & BMSR_100FULL)
features |= SUPPORTED_100baseT_Full;
if (val & BMSR_100HALF)
features |= SUPPORTED_100baseT_Half;
if (val & BMSR_10FULL)
features |= SUPPORTED_10baseT_Full;
if (val & BMSR_10HALF)
features |= SUPPORTED_10baseT_Half;

if (val & BMSR_ESTATEN) {
val = phy_read(phydev, MII_ESTATUS);
if (val < 0)
return val;

if (val & ESTATUS_1000_TFULL)
features |= SUPPORTED_1000baseT_Full;
if (val & ESTATUS_1000_THALF)
features |= SUPPORTED_1000baseT_Half;
}

phydev->supported &= features;
phydev->advertising &= features;

return 0;
}

从代码跟踪来看,通用的PHY驱动没有开启PAUSE功能。为了全面性,又跟踪了Intel的igb驱动,获取网卡信息在igb_get_settings函数中,对于暂停帧处理代码如下:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
switch (hw->fc.requested_mode) {
case e1000_fc_full:
ecmd->advertising |= ADVERTISED_Pause;
break;
case e1000_fc_rx_pause:
ecmd->advertising |= (ADVERTISED_Pause |
ADVERTISED_Asym_Pause);
break;
case e1000_fc_tx_pause:
ecmd->advertising |= ADVERTISED_Asym_Pause;
break;
default:
ecmd->advertising &= ~(ADVERTISED_Pause |
ADVERTISED_Asym_Pause);
}

可以看到,如果为fc_full时,就只赋值ADVERTISED_Pause。这在一定程度上对PAUSE和ASM_DIR的取值进行了验证。

附录

千兆PAUSE优先级解释:
http://www.ieee802.org/3/interp/interp-1-1104.pdf
对PAUSE和ASM_DIR讨论的文章:
http://www.juniper.net/techpubs/en_US/junos13.2/topics/concept/cos-qfx-series-congestion-notification-understanding.html
http://www.tomshardware.com/forum/19497-42-weird-asymmetric-pause-autoconfiguration-code
e1000驱动关于标准文档PAUSE/ASM_DIR的值的说明(约2219行): http://lxr.oss.org.cn/source//drivers/net/ethernet/intel/e1000/e1000_hw.c?v=3.17

李迟 2015.4.11