Skip to content
  • David Gibson's avatar
    kvm: Comparison with ioctl number macros needs to be unsigned · 92e4b519
    David Gibson authored
    
    
    In kvm-all.c we store an ioctl cmd number in the irqchip_inject_ioctl field
    of KVMState, which has type 'int'.  This seems to make sense since the
    ioctl() man page says that the cmd parameter has type int.
    
    However, the kernel treats ioctl numbers as unsigned - sys_ioctl() takes an
    unsigned int, and the macros which generate ioctl numbers expand to
    unsigned expressions.  Furthermore, some ioctls (IOC_READ ioctls on x86
    and IOC_WRITE ioctls on powerpc) have bit 31 set, and so would be negative
    if interpreted as an int. This has the surprising and compile-breaking
    consequence that in kvm_irqchip_set_irq() where we do:
        return (s->irqchip_inject_ioctl == KVM_IRQ_LINE) ? 1 : event.status;
    We will get a "comparison is always false due to limited range of data
    type" warning from gcc if KVM_IRQ_LINE is one of the bit-31-set ioctls,
    which it is on powerpc.
    
    So, despite the fact that the man page and posix say ioctl numbers are
    signed, they're actually unsigned.  The kernel uses unsigned, the glibc
    header uses unsigned long, and FreeBSD, NetBSD and OSX also use unsigned
    long ioctl numbers in the code.
    
    Therefore, this patch changes the variable to be unsigned, fixing the
    compile.
    
    Signed-off-by: default avatarDavid Gibson <david@gibson.dropbear.id.au>
    Signed-off-by: default avatarAlexander Graf <agraf@suse.de>
    92e4b519