云計算之路-阿里云上：消滅“黑色n秒”第一招——不讓CPU空閑

文章出處

昨天對“黑色n秒”問題的最終猜想以失敗而告終，從而讓我們結束了被動猜想階段，進入了主動進攻階段——出招。

今天出第一招——用C#寫個小程序，讓其在每個CPU核上運行一個線程，不讓任何一個CPU核進入空閑（idle）狀態，以進一步排除CPU idle引起的“黑色n秒”。

在這一招中，借助的最重要的武器是System.Diagnostics.ProcessThread.ProcessorAffinity。通過給ProcessorAffinity設置一個掩碼，就可以指定當前線程運行于哪個CPU核上。

如上圖，用哪個核就把那個核對應的二進制位置1，其他位保持0。

所以對于我們所用的8核CPU，從第1核到第8核對應的ProcessorAffinity分別是：1, 2, 4, 8, 16, 32, 64, 128。

需要注意的地方是ProcessThread.ProcessorAffinity針對的是Windows操作系統線程，而.NET線程并不是與操作系統線程一一對應的，一個.NET線程可以運行于多個操作系統線程。所以，如果僅僅指定ProcessThread.ProcessorAffinity，并不能保證.NET線程運行于指定的CPU核上。那怎么辦呢？

微軟提供了解決方案，在設置ProcessThread.ProcessorAffinity之前需要通過下面的代碼將.NET線程固定在操作系統線程上：

Thread.BeginThreadAffinity();

還有一個需要解決的問題是如何讓一個線程一直處于執行狀態，從而不讓其所在的CPU核進入idle狀態。微軟也提供了解決方案，調用非托管方法SetThreadExecutionState()，代碼如下：

NativeMethods.SetThreadExecutionState(NativeMethods.ES_CONTINUOUS | NativeMethods.ES_SYSTEM_REQUIRED);

下面請看招式：

代碼第1部分：

class Program
{
    static void Main(string[] args)
    {
        var threads = new Thread[Environment.ProcessorCount];
        Console.WriteLine("Processor Count:" + Environment.ProcessorCount);
        for (int i = 0; i < threads.Length; i++)
        {
            var coreNumber = i + 1;
            var threaName = "ThreadOnCPU" + coreNumber;
            var start = new ThreadStart(() =>
            {
                Console.WriteLine(threaName + " is working...");
                NativeMethods.SetThreadExecutionState(
                    NativeMethods.ES_CONTINUOUS | NativeMethods.ES_SYSTEM_REQUIRED);
            });
            var thread = new DistributedThread(start);
            thread.ProcessorAffinity = (int)Math.Pow(2, i);
            thread.ManagedThread.Name = threaName;
            thread.Start();
        }
        Console.ReadKey();
    }       
}

internal static class NativeMethods
{
    [DllImport("kernel32.dll")]
    public static extern uint SetThreadExecutionState(uint esFlags);
    public const uint ES_CONTINUOUS = 0x80000000;
    public const uint ES_SYSTEM_REQUIRED = 0x00000001;
}

代碼第2部分（來自Running .NET threads on selected processor cores ）：

class DistributedThread
{
    [DllImport("kernel32.dll")]
    public static extern int GetCurrentThreadId();

    [DllImport("kernel32.dll")]
    public static extern int GetCurrentProcessorNumber();

    private ThreadStart threadStart;

    private ParameterizedThreadStart parameterizedThreadStart;

    private Thread thread;

    public int ProcessorAffinity { get; set; }

    public Thread ManagedThread
    {
        get
        {
            return thread;
        }
    }

    private DistributedThread()
    {
        thread = new Thread(DistributedThreadStart);
    }

    public DistributedThread(ThreadStart threadStart)
        : this()
    {
        this.threadStart = threadStart;
    }

    public DistributedThread(ParameterizedThreadStart threadStart)
        : this()
    {
        this.parameterizedThreadStart = threadStart;
    }

    public void Start()
    {
        if (this.threadStart == null) throw new InvalidOperationException();

        thread.Start(null);
    }

    public void Start(object parameter)
    {
        if (this.parameterizedThreadStart == null) throw new InvalidOperationException();

        thread.Start(parameter);
    }

    private void DistributedThreadStart(object parameter)
    {
        try
        {
            // fix to OS thread
            Thread.BeginThreadAffinity();

            // set affinity
            if (ProcessorAffinity != 0)
            {
                CurrentThread.ProcessorAffinity = new IntPtr(ProcessorAffinity);
            }

            // call real thread
            if (this.threadStart != null)
            {
                this.threadStart();
            }
            else if (this.parameterizedThreadStart != null)
            {
                this.parameterizedThreadStart(parameter);
            }
            else
            {
                throw new InvalidOperationException();
            }
        }
        finally
        {
            // reset affinity
            CurrentThread.ProcessorAffinity = new IntPtr(0xFFFF);
            Thread.EndThreadAffinity();
        }
    }

    private ProcessThread CurrentThread
    {
        get
        {
            int id = GetCurrentThreadId();
            return
                (from ProcessThread th in Process.GetCurrentProcess().Threads
                    where th.Id == id
                    select th).Single();
        }
    }
}