pkt_metadata_prefetch_init() is used to prefetch the packet metadata
before initializing the metadata in pkt_metadata_init(). This is done
for every packet in userspace datapath and is performance critical.

Commit 99fc16c0 prefetches only cachline0 and cacheline2 as the metadata
part of respective cachelines will be initialized by pkt_metadata_init().

However in VXLAN case when popping the vxlan header, netdev_vxlan_pop_header()
invokes pkt_metadata_init_tnl() which zeroes out metadata part of
cacheline1 that wasn't prefetched earlier and causes performance
degradation.

By prefetching cacheline1, 9% performance improvement is observed with
vxlan decapsulation test case for packet sizes of 118 bytes. Performance
variation is observed based on CFLAGS.

       CFLAGS="-O2"                CFLAGS="-O2 -msse4.2"
  Master      4.667 Mpps         Master       4.710 Mpps
  With Patch  5.045 Mpps         With Patch   5.097 Mpps

  CFLAGS="-O2 -march=native"     CFLAGS="-Ofast -march=native"
  Master      5.072 Mpps         Master       5.349 Mpps
  With Patch  5.193 Mpps         With Patch   5.378 Mpps

Fixes: 99fc16c0

 ("Reorganize the pkt_metadata structure.")
Signed-off-by: Bhanuprakash Bodireddy <bhanuprakash.bodireddy@intel.com>
Acked-by: Sugesh Chandran <sugesh.chandran@intel.com>
Signed-off-by: Ben Pfaff <blp@ovn.org>