• Daniel Borkmann's avatar
    bpf: fix checksum fixups on bpf_skb_store_bytes · 479ffccc
    Daniel Borkmann authored
    bpf_skb_store_bytes() invocations above L2 header need BPF_F_RECOMPUTE_CSUM
    flag for updates, so that CHECKSUM_COMPLETE will be fixed up along the way.
    Where we ran into an issue with bpf_skb_store_bytes() is when we did a
    single-byte update on the IPv6 hoplimit despite using BPF_F_RECOMPUTE_CSUM
    flag; simple ping via ICMPv6 triggered a hw csum failure as a result. The
    underlying issue has been tracked down to a buffer alignment issue.
    Meaning, that csum_partial() computations via skb_postpull_rcsum() and
    skb_postpush_rcsum() pair invoked had a wrong result since they operated on
    an odd address for the hoplimit, while other computations were done on an
    even address. This mix doesn't work as-is with skb_postpull_rcsum(),
    skb_postpush_rcsum() pair as it always expects at least half-word alignment
    of input buffers, which is normally the case. Thus, instead of these helpers
    using csum_sub() and (implicitly) csum_add(), we need to use csum_block_sub(),
    csum_block_add(), respectively. For unaligned offsets, they rotate the sum
    to align it to a half-word boundary again, otherwise they work the same as
    csum_sub() and csum_add().
    Adding __skb_postpull_rcsum(), __skb_postpush_rcsum() variants that take the
    offset as an input and adapting bpf_skb_store_bytes() to them fixes the hw
    csum failures again. The skb_postpull_rcsum(), skb_postpush_rcsum() helpers
    use a 0 constant for offset so that the compiler optimizes the offset & 1
    test away and generates the same code as with csum_sub()/_add().
    Fixes: 608cd71a ("tc: bpf: generalize pedit action")
    Signed-off-by: default avatarDaniel Borkmann <daniel@iogearbox.net>
    Acked-by: default avatarAlexei Starovoitov <ast@kernel.org>
    Signed-off-by: default avatarDavid S. Miller <davem@davemloft.net>
skbuff.h 110 KB