Started integrating IPC with Async code. This is not functional yet. There is a bug inside the Async code. It seems to be walking up the stack incorrectly and eventually dereferencing something that is not a valid pointer. Working on fixing it.

Split create_channel into two functions:
create_channel simply allocates space for the channel and
the ring buffer.

attach_thread_to_channel:
This function creates a kthread and pins it to a specific core.
It takes a funciton pointer to the function you want to execute.

**NOTE**
The timing code is still using the API, It will be fixed in future
commits

Signed-off-by: Scott Bauer <sbauer@eng.utah.edu>

Signed-off-by: Scotty Bauer <sbauer@eng.utah.edu>

Signed-off-by: Scotty Bauer <sbauer@eng.utah.edu>

Signed-off-by: Scotty Bauer <sbauer@eng.utah.edu>

Signed-off-by: Scotty Bauer <sbauer@eng.utah.edu>

Signed-off-by: Scotty Bauer <sbauer@eng.utah.edu>

Signed-off-by: Scotty Bauer <sbauer@eng.utah.edu>

TODO:
Another test for timing

Signed-off-by: Scotty Bauer <sbauer@eng.utah.edu>

Signed-off-by: Scotty Bauer <sbauer@eng.utah.edu>

Signed-off-by: Scotty Bauer <sbauer@eng.utah.edu>

Ring-chan modification includes passing a compile-time const size to get rec. Compiler should be able to constant fold it.

Signed-off-by: Scotty Bauer <sbauer@eng.utah.edu>

Signed-off-by: Scotty Bauer <sbauer@eng.utah.edu>

Simple RPC takes parameters and adds them on another core

Signed-off-by: Scotty Bauer <sbauer@eng.utah.edu>

Signed-off-by: Scotty Bauer <sbauer@eng.utah.edu>

Signed-off-by: Scotty Bauer <sbauer@eng.utah.edu>

Signed-off-by: Scotty Bauer <sbauer@eng.utah.edu>

IPC:
IPC needs to be its own stand-alone set of functions which just do IPC. Before in kernel_rpc as well as in the kernel/betamodule set of misc drivers the tests were incorporated into the IPC code itself. It was then driven from a userland program. In reality this isn't anywhere close to what we need to accomplish. Because of this we abstract all the true IPC code into its own .o file which can be linked in and used in future modules.

ring-chan:
Due to the re-write I've started re-writing the ring-channel code as well as the actual implementation of the ring-channels. There were way too many variables we were not using as well as levels of pointer chasing to get sizes etc which is unecessary for us, and slow.

TODO:
Finish ring-chan cleanup
rewrite make files
setup tests dir -- essentially an RPC mechanism
research intel HLE to see if it holds the key to our ~200cycle IPC on a single line

Signed-off-by: Scotty Bauer <sbauer@eng.utah.edu>

I need to work out a few synchronization bugs

Signed-off-by: Scotty Bauer <sbauer@eng.utah.edu>

Signed-off-by: Scotty Bauer <sbauer@eng.utah.edu>

This setup is for FLOOD tests with pause instruction, CPU is set @ 2.4ghz back down to what userland produces, 320-380 cycles, all the way to 60 cycles Round trip for 64 message flood

Signed-off-by: Scotty Bauer <sbauer@eng.utah.edu>

Added two files, they ping pong a uint64 through the same cache line Slave on CPU1 master on CPU3 results:

MIN     MAX     AVG     MEDIAN
252 & 5704 & 274 & 272

Signed-off-by: Scotty Bauer <sbauer@eng.utah.edu>

Added Two files to the cost-of-synch which is the _SAME_ implementation as what we see in kernel land. Times for ping-pong in both implementaitons.

Kerneland: [ 4861.552684] 378   178176    471     456
Userland:                 MIN     MAX     AVG     MEDIAN
                          312    21424    460     376

Signed-off-by: Scotty Bauer <sbauer@eng.utah.edu>

Fixed bug in make file. cpu_relax() now actually gets compiled into pause latency drops ~30-40 cycles, new numbers incoming

Signed-off-by: Scotty Bauer <sbauer@eng.utah.edu>

Signed-off-by: Scott Bauer <sbauer@node0.xcap-ipc.fastnet.apt.emulab.net>

Change thread creation to functions the vanilla kernel can use, removed kthread_create_on_cpu as well as kthread_unpark, both require specialized compiled kernel

Signed-off-by: Scotty Bauer <sbauer@eng.utah.edu>

Fixed bug in USE_FLOOD Macro that didn't properly update the slot we're supposed to be writing into.

Test is 10k floods of 32 messages. BetaModule1 uses slow math (Imul/divu) while beta2 is using the fast math (and shl)

Test1:
Apr  3 13:42:01 localhost kernel: [  487.326686] MIN    MAX     AVG     MEDIAN
Apr  3 13:42:01 localhost kernel: [  487.329795] 3244 & 144184 & 3728 & 3692
Median time normalized per msg:115 cycles

Test2:
Apr  3 13:43:41 localhost kernel: [  587.197275] MIN    MAX     AVG     MEDIAN
Apr  3 13:43:41 localhost kernel: [  587.200322] 4476 & 318180 & 4888 & 4794
Median time noramlized per msg:149cycles

Test3:
Apr  3 13:44:14 localhost kernel: [  620.868921] MIN    MAX     AVG     MEDIAN
Apr  3 13:44:14 localhost kernel: [  620.872026] 4176 & 156498 & 4613 & 4511
Median time normalized per msg:140Cycles

Signed-off-by: Scotty Bauer <sbauer@eng.utah.edu>

Signed-off-by: Scotty Bauer <sbauer@eng.utah.edu>

This test we remove responses, this is esentially a one way message. We spam 10k messages and see how long it takes to place flag. We're trying to deduce where the cycles are coming from.

TEST1
[ 2929.510089] MIN      MAX     AVG     MEDIAN
[ 2929.513137] 44 & 160 & 47 & 48
TEST2
[ 3354.965156] MIN      MAX     AVG     MEDIAN
[ 3354.968268] 55 & 149 & 63 & 62
TEST3
[ 3531.555669] MIN      MAX     AVG     MEDIAN
[ 3531.558715] 55 & 7199 & 64 & 62

 740:   0f a2                   cpuid
 742:   0f 31                   rdtsc
 744:   41 89 d2                mov    %edx,%r10d
 747:   41 89 c1                mov    %eax,%r9d
 74a:   49 c1 e2 20             shl    bashx20,%r10
 74e:   45 89 c9                mov    %r9d,%r9d
 751:   4d 09 ca                or     %r9,%r10
 754:   8b 46 3c                mov    0x3c(%rsi),%eax
 757:   3d ad 6b 34 c1          cmp    bashxc1346bad,%eax
 75c:   75 f6                   jne    754 <ipc_thread_func+0x94>
 75e:   c7 46 3c ef be ad 0b    movl   bashxbadbeef,0x3c(%rsi)
 765:   0f 01 f9                rdtscp

Signed-off-by: Scotty Bauer <sbauer@eng.utah.edu>

Test is 10k messages, Before writing into the producer buffers they check to see if they're allowed to write into the buffer. TIming is done with seralizing rdtsc then rdtscp per message.

TEST1
[ 2367.106619] MIN      MAX     AVG     MEDIAN
[ 2367.109727] 474 & 82278 & 524 & 510
TEST2
[ 2410.071404] MIN      MAX     AVG     MEDIAN
[ 2410.074514] 468 & 21774 & 514 & 510
TEST3
[ 2475.564114] MIN      MAX     AVG     MEDIAN
[ 2475.567282] 474 & 152550 & 528 & 510

Signed-off-by: Scotty Bauer <sbauer@eng.utah.edu>

BetaModule and BetaModule2 **NOTICE PREVIOUS NUMBERS REPORTED IN COMMITS PREVIOUS TO THIS ONE ARE INCORRECT**

Ping pong of 10k messages. Both sides do _NOT_ check if there is a producer message slot avaiable to write, they just write into it. Timing is done PER message via serializing rdtsc and rdtscp.

Times:
TEST1
[ 1151.501229] MIN      MAX     AVG     MEDIAN
[ 1151.504313] 468 & 131184 & 513 & 504
TEST2
[ 1189.250206] MIN      MAX     AVG     MEDIAN
[ 1189.253377] 437 & 1358 & 478 & 474
TEST3
[ 1221.812533] MIN      MAX     AVG     MEDIAN
[ 1221.815580] 442 & 12420 & 500 & 498

Signed-off-by: Scotty Bauer <sbauer@eng.utah.edu>

Test is 10k messages, 256 page ring buffer, Both sides do not Ack the message meaning the slot cannot be re-used. Since we're only sending 10k 64 byte messages we never wrap around the  buffer.

Timed using seralized RDTSC as start and RDTSCP as finish:

TEST1
[  517.377994] MIN      MAX     AVG     MEDIAN
[  517.381168] 514 & 14780 & 561 & 556
TEST2
[  589.036190] MIN      MAX     AVG     MEDIAN
[  589.039295] 483 & 8091 & 531 & 525
TEST3
[  618.715175] MIN      MAX     AVG     MEDIAN
[  618.718222] 514 & 2299 & 556 & 554

Signed-off-by: Scotty Bauer <sbauer@eng.utah.edu>

10k messages, prodcer/copnsumer do not ack timing outside of loop

Mar 31 17:21:45 localhost kernel: [ 2745.399627] 5856822
Mar 31 17:22:52 localhost kernel: [ 2813.144249] 5651424
Mar 31 17:22:57 localhost kernel: [ 2818.273022] 5227483
Mar 31 17:23:03 localhost kernel: [ 2824.320729] 5652252
Mar 31 17:23:18 localhost kernel: [ 2839.257694] 5813550
Above are cycles for 10k messages ~ 585cycles per message

Signed-off-by: Scotty Bauer <sbauer@eng.utah.edu>

reafactored module 2, This refacotring is slower because we're now missing l1 cache instructions I believe.

Signed-off-by: Scott Bauer <sbauer@node0.xcap-ipc.xcap.emulab.net>

Signed-off-by: Scott Bauer <sbauer@node0.xcap-ipc.xcap.emulab.net>

Signed-off-by: Scott Bauer <sbauer@node0.xcap-ipc.xcap.emulab.net>