The iommu apis did not have provisions to set the non-cacheable bit. We
tracked this down when the values we set in ixgbe RX registers were
disappearing after a while.

Also change the API for accessing priv_pool

Create two LCD modules for ixgbe smp. On the LCD side, create two
hardware queue vectors s.t each LCD module takes care of polling
one q_vector (and a pair of tx/rx queue tied to that q_vector).

Use a local variable instead of a monotonically increasing counter for
number of lcds.

setup_once, iperf:9948 lcdenter | skbqueue 1
setup_once, iperf:9949 lcdenter | skbqueue 0
message from lcd ffff882821c7ac00: msg @ /local/devel/lcd-smp/lcd-domains/test_mods/nullnet/dummy_lcd/glue/dispatch.c:68:
message from lcd ffff882821c7ac00: net got PREP_CHANNEL msg
message from lcd ffff882821f7f800: msg @ /local/devel/lcd-smp/lcd-domains/test_mods/nullnet/dummy_lcd/glue/dispatch.c:68:
message from lcd ffff882821c7ac00: end of msg
message from lcd ffff882821f7f800: net got PREP_CHANNEL msg
message from lcd ffff882821f7f800: end of msg
message from lcd ffff882821c7ac00: msg @ /local/devel/lcd-smp/lcd-domains/test_mods/nullnet/dummy_lcd/glue/nullnet_caller.c:242:
message from lcd ffff882821c7ac00: setup_async_channel

message from lcd ffff882821f7f800: msg @ /local/devel/lcd-smp/lcd-domains/test_mods/nullnet/dummy_lcd/glue/nullnet_caller.c:242:
message from lcd ffff882821c7ac00: end of msg
message from lcd ffff882821f7f800: setup_async_channel

message from lcd ffff882821f7f800: end of msg
message from lcd ffff882821c7ac00: msg @ /local/devel/lcd-smp/lcd-domains/test_mods/nullnet/dummy_lcd/glue/nullnet_caller.c:284:
message from lcd ffff882821f7f800: msg @ /local/devel/lcd-smp/lcd-domains/test_mods/nullnet/dummy_lcd/glue/nullnet_caller.c:284:
message from lcd ffff882821c7ac00: ==> Prep buffers
message from lcd ffff882821c7ac00: end of msg
message from lcd ffff882821f7f800: ==> Prep buffers
message from lcd ffff882821f7f800: end of msg
message from lcd ffff882821c7ac00: msg @ /local/devel/lcd-smp/lcd-domains/test_mods/nullnet/dummy_lcd/glue/nullnet_caller.c:1135:
message from lcd ffff882821f7f800: msg @ /local/devel/lcd-smp/lcd-domains/test_mods/nullnet/dummy_lcd/glue/nullnet_caller.c:1135:
message from lcd ffff882821c7ac00: prep_channel_callee: Preparing sync send
message from lcd ffff882821f7f800: prep_channel_callee: Preparing sync send
message from lcd ffff882821c7ac00: end of msg
message from lcd ffff882821f7f800: end of msg

Tested nullnet with two iperf processes, each sending packets via
different LCDs.
[  9958] 0.00-60.00 sec  56.0 GBytes  8.02 Gbits/sec  40895407/0   681589 pps
[  9958] Sent 40895407 datagrams
[  9957] 0.00-60.00 sec  56.0 GBytes  8.01 Gbits/sec  40891131/0   681518 pps
[  9957] Sent 40891131 datagrams

Create current_task for child LCD

For each LCD, maintain a current pointer to store PTS() and async
related stuff.

Roll-in spinlocks from xv6 as Linux spinlocks has a ton of dependencies.
Bring back the elided lock definitions.

Bring in support for registration of async channel from the child LCD.
Make sure all instances of async channel communication knows with
channel they are communicating with KLCD (tag it with LCD_ID).

As of now, the child LCD boots, creates an async communication channel
with KLCD and spins on a tight while loop. From the below trimmed log,
one can notice that we have two LCDs (note the LCD pointer at message
for lcd xxxxx)

message from lcd ffff8813fde63c00: msg @ /local/devel/lcd-smp/lcd-domains/test_mods/nullnet/dummy_lcd/main.c:218:
message from lcd ffff8813fde63c00: __dummy_lcd_init: entering
message from lcd ffff8813fde63c00: LCD 0 entering...
message from lcd ffff8813fde63c00: msg @ /local/devel/lcd-smp/lcd-domains/liblcd/lcd-domains/enter_exit.c:60:
message from lcd ffff8813fde63c00: call endpoint created and installed
...
message from lcd ffff8813fde63c00: ===============
message from lcd ffff8813fde63c00:   LCD BOOTED
message from lcd ffff8813fde63c00: ===============
message from lcd ffff8813fde63c00: [0] nullnet reg channel 261
...
message from klcd ffff8813fde66000: Starting LCD 1
...
message from lcd ffff8813fde64000: LCD 1 entering...
message from lcd ffff8813fde64000: ===============
message from lcd ffff8813fde64000:   Child LCD BOOTED
message from lcd ffff8813fde64000: ===============
...
message from lcd ffff8813fde63c00: SUCCESSFULLY REGISTERED DUMMY!
message from klcd ffff8813fde65800: register_child child 1 registration received, setting up thc_chl
message from klcd ffff8813fde65800: msg @ /local/devel/lcd-smp/lcd-domains/test_mods/nullnet/net_klcd/glue/nullnet_callee.c:2152:
message from klcd ffff8813fde65800: register_child, child 1 registration complete

Introduce per-lcd global segment page where each lcd will have its
boot_info. Modify lcd_get_boot_info to pull the page from gs instead of
a hardcoded address. Now, only gs:0x0 is populated. In the future, if
needed, we can have other private per-lcd variables as well.

Earlier the gs base was set to NULL. Now allocate a page
and configure lcd to use that page for saving and retrieving
the stack canary. Apart from stack canaries gs can be used for other
purposes as well (Such as percpu).

Partition bootstrap pages, stack pages s.t each LCD will use its own
set of pages.

As we map the same code/data pages of the LCD module in both the EPTs
(parent and child), make sure to avoid double-free

Enable the parent LCD to proceed with module init whereas the child LCD
is spinning on a while loop.

Modify dummy_lcd to just boot, init and spin on a while loop.