C++11 function static variable initialization weirdness - multithreading

Question

I have plain simple function that maps some HTTP status codes to strings:
const std::string & status_name(int code)
{
static std::unordered_map<int, std::string> status_table {
{ 0, "UNKNOWN" },
{ 100, "Continue" },
{ 101, "Switching Protocols" },
{ 200, "OK" },
{ 201, "Created" },
{ 202, "Accepted" },
{ 203, "Non-Authoritative Information" },
{ 204, "No Content" },
{ 205, "Reset Content" },
{ 206, "Partial Content" },
{ 300, "Multiple Choices" },
{ 301, "Moved Permanently" },
{ 302, "Found" },
{ 303, "See Other" },
{ 304, "Not Modified" },
{ 305, "Use Proxy" },
{ 307, "Temporary Redirect" },
{ 400, "Bad Request" },
{ 401, "Unauthorized" },
{ 402, "Payment Required" },
{ 403, "Forbidden" },
{ 404, "Not Found" },
{ 405, "Method Not Allowed" },
{ 406, "Not Acceptable" },
{ 407, "Proxy Authentication Required" },
{ 408, "Request Timeout" },
{ 409, "Conflict" },
{ 410, "Gone" },
{ 411, "Length Required" },
{ 412, "Precondition Failed" },
{ 413, "Request Entity Too Large" },
{ 414, "Request-URI Too Long" },
{ 415, "Unsupported Media Type" },
{ 416, "Requested Range Not Satisfiable" },
{ 417, "Expectation Failed" },
{ 499, "Rotten session" },
{ 500, "Internal Server Error" },
{ 501, "Not Implemented" },
{ 502, "Bad Gateway" },
{ 503, "Service Unavailable" },
{ 504, "Gateway Timeout" },
{ 505, "HTTP Version Not Supported" },
};
auto it = status_table.find(code);
if (it != status_table.end())
return it->second;
return status_table.at(0);
}
One day, on our test servers, a crash has occurred. Analysing core dump is seems that std::out_of_range exception was thrown in status_table.at(0);. Another weird thing is that status code in question was 200. Program that crashed abuses multiple threads, so my stomach feel tells me that this issue is thread-related.
I'm really puzzled about how it's even possible. It's C++11 and this version of standard provides guarantees regarding thread-safety of static variable initialization. The compiler is clang-3.3, architecture is x64 and CPU is Intel(R) Xeon(R) CPU E5506.
The only two things come to mind:
there's some bug in the code I miss to see;
this weirdness is related to CPU cache incoherency: thread that caused crash saw guard variable as indicating 'initialized' state of
status_table but object itself was seen empty.
Update on the issue:
After making a small test case I was able to easily reproduce the problem:
#include <thread>
#include <atomic>
#include <string>
#include <unordered_map>
#include <vector>
#include <memory>
#include <iostream>
#include <unistd.h>
const std::string & status_name1(int code)
{
static std::unordered_map<int, std::string> status_table {
{ 0, "UNKNOWN" },
{ 100, "Continue" },
{ 101, "Switching Protocols" },
{ 200, "OK" },
{ 201, "Created" },
{ 202, "Accepted" },
{ 203, "Non-Authoritative Information" },
{ 204, "No Content" },
{ 205, "Reset Content" },
{ 206, "Partial Content" },
{ 300, "Multiple Choices" },
{ 301, "Moved Permanently" },
{ 302, "Found" },
{ 303, "See Other" },
{ 304, "Not Modified" },
{ 305, "Use Proxy" },
{ 307, "Temporary Redirect" },
{ 400, "Bad Request" },
{ 401, "Unauthorized" },
{ 402, "Payment Required" },
{ 403, "Forbidden" },
{ 404, "Not Found" },
{ 405, "Method Not Allowed" },
{ 406, "Not Acceptable" },
{ 407, "Proxy Authentication Required" },
{ 408, "Request Timeout" },
{ 409, "Conflict" },
{ 410, "Gone" },
{ 411, "Length Required" },
{ 412, "Precondition Failed" },
{ 413, "Request Entity Too Large" },
{ 414, "Request-URI Too Long" },
{ 415, "Unsupported Media Type" },
{ 416, "Requested Range Not Satisfiable" },
{ 417, "Expectation Failed" },
{ 499, "Rotten session" },
{ 500, "Internal Server Error" },
{ 501, "Not Implemented" },
{ 502, "Bad Gateway" },
{ 503, "Service Unavailable" },
{ 504, "Gateway Timeout" },
{ 505, "HTTP Version Not Supported" },
};
auto it = status_table.find(code);
if (it != status_table.end())
return it->second;
return status_table.at(0);
}
const std::string & status_name2(int code)
{
static std::unordered_map<int, std::string> status_table {
{ 0, "UNKNOWN" },
{ 100, "Continue" },
{ 101, "Switching Protocols" },
{ 200, "OK" },
{ 201, "Created" },
{ 202, "Accepted" },
{ 203, "Non-Authoritative Information" },
{ 204, "No Content" },
{ 205, "Reset Content" },
{ 206, "Partial Content" },
{ 300, "Multiple Choices" },
{ 301, "Moved Permanently" },
{ 302, "Found" },
{ 303, "See Other" },
{ 304, "Not Modified" },
{ 305, "Use Proxy" },
{ 307, "Temporary Redirect" },
{ 400, "Bad Request" },
{ 401, "Unauthorized" },
{ 402, "Payment Required" },
{ 403, "Forbidden" },
{ 404, "Not Found" },
{ 405, "Method Not Allowed" },
{ 406, "Not Acceptable" },
{ 407, "Proxy Authentication Required" },
{ 408, "Request Timeout" },
{ 409, "Conflict" },
{ 410, "Gone" },
{ 411, "Length Required" },
{ 412, "Precondition Failed" },
{ 413, "Request Entity Too Large" },
{ 414, "Request-URI Too Long" },
{ 415, "Unsupported Media Type" },
{ 416, "Requested Range Not Satisfiable" },
{ 417, "Expectation Failed" },
{ 499, "Rotten session" },
{ 500, "Internal Server Error" },
{ 501, "Not Implemented" },
{ 502, "Bad Gateway" },
{ 503, "Service Unavailable" },
{ 504, "Gateway Timeout" },
{ 505, "HTTP Version Not Supported" },
};
auto it = status_table.find(code);
if (it != status_table.end())
return it->second;
return status_table.at(0);
}
const std::string & status_name3(int code)
{
static std::unordered_map<int, std::string> status_table {
{ 0, "UNKNOWN" },
{ 100, "Continue" },
{ 101, "Switching Protocols" },
{ 200, "OK" },
{ 201, "Created" },
{ 202, "Accepted" },
{ 203, "Non-Authoritative Information" },
{ 204, "No Content" },
{ 205, "Reset Content" },
{ 206, "Partial Content" },
{ 300, "Multiple Choices" },
{ 301, "Moved Permanently" },
{ 302, "Found" },
{ 303, "See Other" },
{ 304, "Not Modified" },
{ 305, "Use Proxy" },
{ 307, "Temporary Redirect" },
{ 400, "Bad Request" },
{ 401, "Unauthorized" },
{ 402, "Payment Required" },
{ 403, "Forbidden" },
{ 404, "Not Found" },
{ 405, "Method Not Allowed" },
{ 406, "Not Acceptable" },
{ 407, "Proxy Authentication Required" },
{ 408, "Request Timeout" },
{ 409, "Conflict" },
{ 410, "Gone" },
{ 411, "Length Required" },
{ 412, "Precondition Failed" },
{ 413, "Request Entity Too Large" },
{ 414, "Request-URI Too Long" },
{ 415, "Unsupported Media Type" },
{ 416, "Requested Range Not Satisfiable" },
{ 417, "Expectation Failed" },
{ 499, "Rotten session" },
{ 500, "Internal Server Error" },
{ 501, "Not Implemented" },
{ 502, "Bad Gateway" },
{ 503, "Service Unavailable" },
{ 504, "Gateway Timeout" },
{ 505, "HTTP Version Not Supported" },
};
auto it = status_table.find(code);
if (it != status_table.end())
return it->second;
return status_table.at(0);
}
const std::string & status_name4(int code)
{
static std::unordered_map<int, std::string> status_table {
{ 0, "UNKNOWN" },
{ 100, "Continue" },
{ 101, "Switching Protocols" },
{ 200, "OK" },
{ 201, "Created" },
{ 202, "Accepted" },
{ 203, "Non-Authoritative Information" },
{ 204, "No Content" },
{ 205, "Reset Content" },
{ 206, "Partial Content" },
{ 300, "Multiple Choices" },
{ 301, "Moved Permanently" },
{ 302, "Found" },
{ 303, "See Other" },
{ 304, "Not Modified" },
{ 305, "Use Proxy" },
{ 307, "Temporary Redirect" },
{ 400, "Bad Request" },
{ 401, "Unauthorized" },
{ 402, "Payment Required" },
{ 403, "Forbidden" },
{ 404, "Not Found" },
{ 405, "Method Not Allowed" },
{ 406, "Not Acceptable" },
{ 407, "Proxy Authentication Required" },
{ 408, "Request Timeout" },
{ 409, "Conflict" },
{ 410, "Gone" },
{ 411, "Length Required" },
{ 412, "Precondition Failed" },
{ 413, "Request Entity Too Large" },
{ 414, "Request-URI Too Long" },
{ 415, "Unsupported Media Type" },
{ 416, "Requested Range Not Satisfiable" },
{ 417, "Expectation Failed" },
{ 499, "Rotten session" },
{ 500, "Internal Server Error" },
{ 501, "Not Implemented" },
{ 502, "Bad Gateway" },
{ 503, "Service Unavailable" },
{ 504, "Gateway Timeout" },
{ 505, "HTTP Version Not Supported" },
};
auto it = status_table.find(code);
if (it != status_table.end())
return it->second;
return status_table.at(0);
}
const std::string & status_name5(int code)
{
static std::unordered_map<int, std::string> status_table {
{ 0, "UNKNOWN" },
{ 100, "Continue" },
{ 101, "Switching Protocols" },
{ 200, "OK" },
{ 201, "Created" },
{ 202, "Accepted" },
{ 203, "Non-Authoritative Information" },
{ 204, "No Content" },
{ 205, "Reset Content" },
{ 206, "Partial Content" },
{ 300, "Multiple Choices" },
{ 301, "Moved Permanently" },
{ 302, "Found" },
{ 303, "See Other" },
{ 304, "Not Modified" },
{ 305, "Use Proxy" },
{ 307, "Temporary Redirect" },
{ 400, "Bad Request" },
{ 401, "Unauthorized" },
{ 402, "Payment Required" },
{ 403, "Forbidden" },
{ 404, "Not Found" },
{ 405, "Method Not Allowed" },
{ 406, "Not Acceptable" },
{ 407, "Proxy Authentication Required" },
{ 408, "Request Timeout" },
{ 409, "Conflict" },
{ 410, "Gone" },
{ 411, "Length Required" },
{ 412, "Precondition Failed" },
{ 413, "Request Entity Too Large" },
{ 414, "Request-URI Too Long" },
{ 415, "Unsupported Media Type" },
{ 416, "Requested Range Not Satisfiable" },
{ 417, "Expectation Failed" },
{ 499, "Rotten session" },
{ 500, "Internal Server Error" },
{ 501, "Not Implemented" },
{ 502, "Bad Gateway" },
{ 503, "Service Unavailable" },
{ 504, "Gateway Timeout" },
{ 505, "HTTP Version Not Supported" },
};
auto it = status_table.find(code);
if (it != status_table.end())
return it->second;
return status_table.at(0);
}
const std::string & status_name6(int code)
{
static std::unordered_map<int, std::string> status_table {
{ 0, "UNKNOWN" },
{ 100, "Continue" },
{ 101, "Switching Protocols" },
{ 200, "OK" },
{ 201, "Created" },
{ 202, "Accepted" },
{ 203, "Non-Authoritative Information" },
{ 204, "No Content" },
{ 205, "Reset Content" },
{ 206, "Partial Content" },
{ 300, "Multiple Choices" },
{ 301, "Moved Permanently" },
{ 302, "Found" },
{ 303, "See Other" },
{ 304, "Not Modified" },
{ 305, "Use Proxy" },
{ 307, "Temporary Redirect" },
{ 400, "Bad Request" },
{ 401, "Unauthorized" },
{ 402, "Payment Required" },
{ 403, "Forbidden" },
{ 404, "Not Found" },
{ 405, "Method Not Allowed" },
{ 406, "Not Acceptable" },
{ 407, "Proxy Authentication Required" },
{ 408, "Request Timeout" },
{ 409, "Conflict" },
{ 410, "Gone" },
{ 411, "Length Required" },
{ 412, "Precondition Failed" },
{ 413, "Request Entity Too Large" },
{ 414, "Request-URI Too Long" },
{ 415, "Unsupported Media Type" },
{ 416, "Requested Range Not Satisfiable" },
{ 417, "Expectation Failed" },
{ 499, "Rotten session" },
{ 500, "Internal Server Error" },
{ 501, "Not Implemented" },
{ 502, "Bad Gateway" },
{ 503, "Service Unavailable" },
{ 504, "Gateway Timeout" },
{ 505, "HTTP Version Not Supported" },
};
auto it = status_table.find(code);
if (it != status_table.end())
return it->second;
return status_table.at(0);
}
const std::string & status_name7(int code)
{
static std::unordered_map<int, std::string> status_table {
{ 0, "UNKNOWN" },
{ 100, "Continue" },
{ 101, "Switching Protocols" },
{ 200, "OK" },
{ 201, "Created" },
{ 202, "Accepted" },
{ 203, "Non-Authoritative Information" },
{ 204, "No Content" },
{ 205, "Reset Content" },
{ 206, "Partial Content" },
{ 300, "Multiple Choices" },
{ 301, "Moved Permanently" },
{ 302, "Found" },
{ 303, "See Other" },
{ 304, "Not Modified" },
{ 305, "Use Proxy" },
{ 307, "Temporary Redirect" },
{ 400, "Bad Request" },
{ 401, "Unauthorized" },
{ 402, "Payment Required" },
{ 403, "Forbidden" },
{ 404, "Not Found" },
{ 405, "Method Not Allowed" },
{ 406, "Not Acceptable" },
{ 407, "Proxy Authentication Required" },
{ 408, "Request Timeout" },
{ 409, "Conflict" },
{ 410, "Gone" },
{ 411, "Length Required" },
{ 412, "Precondition Failed" },
{ 413, "Request Entity Too Large" },
{ 414, "Request-URI Too Long" },
{ 415, "Unsupported Media Type" },
{ 416, "Requested Range Not Satisfiable" },
{ 417, "Expectation Failed" },
{ 499, "Rotten session" },
{ 500, "Internal Server Error" },
{ 501, "Not Implemented" },
{ 502, "Bad Gateway" },
{ 503, "Service Unavailable" },
{ 504, "Gateway Timeout" },
{ 505, "HTTP Version Not Supported" },
};
auto it = status_table.find(code);
if (it != status_table.end())
return it->second;
return status_table.at(0);
}
const std::string & status_name8(int code)
{
static std::unordered_map<int, std::string> status_table {
{ 0, "UNKNOWN" },
{ 100, "Continue" },
{ 101, "Switching Protocols" },
{ 200, "OK" },
{ 201, "Created" },
{ 202, "Accepted" },
{ 203, "Non-Authoritative Information" },
{ 204, "No Content" },
{ 205, "Reset Content" },
{ 206, "Partial Content" },
{ 300, "Multiple Choices" },
{ 301, "Moved Permanently" },
{ 302, "Found" },
{ 303, "See Other" },
{ 304, "Not Modified" },
{ 305, "Use Proxy" },
{ 307, "Temporary Redirect" },
{ 400, "Bad Request" },
{ 401, "Unauthorized" },
{ 402, "Payment Required" },
{ 403, "Forbidden" },
{ 404, "Not Found" },
{ 405, "Method Not Allowed" },
{ 406, "Not Acceptable" },
{ 407, "Proxy Authentication Required" },
{ 408, "Request Timeout" },
{ 409, "Conflict" },
{ 410, "Gone" },
{ 411, "Length Required" },
{ 412, "Precondition Failed" },
{ 413, "Request Entity Too Large" },
{ 414, "Request-URI Too Long" },
{ 415, "Unsupported Media Type" },
{ 416, "Requested Range Not Satisfiable" },
{ 417, "Expectation Failed" },
{ 499, "Rotten session" },
{ 500, "Internal Server Error" },
{ 501, "Not Implemented" },
{ 502, "Bad Gateway" },
{ 503, "Service Unavailable" },
{ 504, "Gateway Timeout" },
{ 505, "HTTP Version Not Supported" },
};
auto it = status_table.find(code);
if (it != status_table.end())
return it->second;
return status_table.at(0);
}
int main(int argc, char ** argv)
{
std::vector<std::thread> thread_pool;
auto guard_variable = std::make_shared<std::atomic<int>>(0);
for (int i = 0; i < 8; ++i) {
thread_pool.push_back(std::thread([guard_variable] {
while (!guard_variable->load(std::memory_order_relaxed)) {
/// Noop.
}
int i = 0;
i += status_name1(200).length();
i += status_name2(200).length();
i += status_name3(200).length();
i += status_name4(200).length();
i += status_name5(200).length();
i += status_name6(200).length();
i += status_name7(200).length();
i += status_name8(200).length();
std::cout << i << std::endl;
}));
}
usleep(50000);
guard_variable->store(1, std::memory_order_relaxed);
for (auto && thread : thread_pool) {
thread.join();
}
return 0;
}
This code works as expected everywhere else other than on that test machine. No matter what version of clang I use (3.3, 3.6) this test program crashes due to exception immediately, at different times in different status_name* function.
What can possibly cause this behaviour? Is CPU cache inconsistency or is it bug in libc++?

Answer 1

Clang supports "magic statics" (N2660 Dynamic initialization and destruction with concurrency) since 2.9 per http://clang.llvm.org/cxx_status.html.
Rather than a bug in the compiler or your immediate code, it is more likely that a bug elsewhere in your code is corrupting program state, causing status_table to appear initialized but empty.