daemon-agent: double free

Agent fails since a few days

=================================================================
==168118==ERROR: AddressSanitizer: attempting double-free on 0x615000898580 in thread T0:
    #0 0x7f6efe80651f in operator delete(void*) ../../../../src/libsanitizer/asan/asan_new_delete.cc:165
    #1 0x7f6efcb65d2b in __gnu_cxx::new_allocator<std::function<void ()> >::deallocate(std::function<void ()>*, unsigned long) /usr/include/c++/9/ext/new_allocator.h:128
    #2 0x7f6efcb65d2b in std::allocator_traits<std::allocator<std::function<void ()> > >::deallocate(std::allocator<std::function<void ()> >&, std::function<void ()>*, unsigned long) /usr/include/c++/9/bits/alloc_traits.h:469
    #3 0x7f6efcb65d2b in std::_Deque_base<std::function<void ()>, std::allocator<std::function<void ()> > >::_M_deallocate_node(std::function<void ()>*) /usr/include/c++/9/bits/stl_deque.h:627
    #4 0x7f6efcb65d2b in std::_Deque_base<std::function<void ()>, std::allocator<std::function<void ()> > >::_M_destroy_nodes(std::function<void ()>**, std::function<void ()>**) /usr/include/c++/9/bits/stl_deque.h:761
    #5 0x7f6efcb65d2b in std::_Deque_base<std::function<void ()>, std::allocator<std::function<void ()> > >::~_Deque_base() /usr/include/c++/9/bits/stl_deque.h:683
    #6 0x7f6efcb65d2b in std::deque<std::function<void ()>, std::allocator<std::function<void ()> > >::~deque() /usr/include/c++/9/bits/stl_deque.h:1072
    #7 0x7f6efcb65d2b in std::queue<std::function<void ()>, std::deque<std::function<void ()>, std::allocator<std::function<void ()> > > >::~queue() /usr/include/c++/9/bits/stl_queue.h:96
    #8 0x7f6efcb65d2b in dht::ThreadPool::stop() /foo/contrib/native/opendht/src/thread_pool.cpp:109
    #9 0x7f6efcb65db6 in dht::ThreadPool::join() /foo/contrib/native/opendht/src/thread_pool.cpp:116
    #10 0x7f6efbe54ae3 in jami::Manager::finish() /foo/src/manager.cpp:902
    #11 0x7f6efbfe6d01 in DRing::fini() /foo/src/ring_api.cpp:99
    #12 0x7f6efb01d8a6  (/lib/x86_64-linux-gnu/libc.so.6+0x468a6)
    #13 0x7f6efb01da5f in exit (/lib/x86_64-linux-gnu/libc.so.6+0x46a5f)
    #14 0x5597e4a2b8aa in c_handler /foo/contrib/native/guile/libguile/continuations.c:441
    #15 0x5597e49f757c in catch_post_unwind_handler /foo/contrib/native/guile/libguile/throw.c:118
    #16 0x5597e49873e0 in scm_c_with_exception_handler /foo/contrib/native/guile/libguile/exceptions.c:167
    #17 0x5597e49f779c in scm_c_catch /foo/contrib/native/guile/libguile/throw.c:168
    #18 0x5597e4a2b69b in scm_i_with_continuation_barrier /foo/contrib/native/guile/libguile/continuations.c:368
    #19 0x5597e4a2b979 in scm_c_with_continuation_barrier /foo/contrib/native/guile/libguile/continuations.c:464
    #20 0x5597e49f53da in with_guile /foo/contrib/native/guile/libguile/threads.c:645
    #21 0x7f6efe6484c1 in GC_call_with_stack_base extra/../misc.c:2106
    #22 0x5597e49f54d3 in scm_i_with_guile /foo/contrib/native/guile/libguile/threads.c:688
    #23 0x5597e49f5517 in scm_with_guile /foo/contrib/native/guile/libguile/threads.c:694
    #24 0x5597e48bf474 in main src/main.cpp:94
    #25 0x7f6efaffb082 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x24082)
    #26 0x5597e48beded in _start (/foo/test/agent/.libs/agent.exe+0xc8ded)

0x615000898580 is located 0 bytes inside of 512-byte region [0x615000898580,0x615000898780)
freed by thread T0 here:
    #0 0x7f6efe80651f in operator delete(void*) ../../../../src/libsanitizer/asan/asan_new_delete.cc:165
    #1 0x7f6efcb65f94 in __gnu_cxx::new_allocator<std::function<void ()> >::deallocate(std::function<void ()>*, unsigned long) /usr/include/c++/9/ext/new_allocator.h:128
    #2 0x7f6efcb65f94 in std::allocator_traits<std::allocator<std::function<void ()> > >::deallocate(std::allocator<std::function<void ()> >&, std::function<void ()>*, unsigned long) /usr/include/c++/9/bits/alloc_traits.h:469
    #3 0x7f6efcb65f94 in std::_Deque_base<std::function<void ()>, std::allocator<std::function<void ()> > >::_M_deallocate_node(std::function<void ()>*) /usr/include/c++/9/bits/stl_deque.h:627
    #4 0x7f6efcb65f94 in std::_Deque_base<std::function<void ()>, std::allocator<std::function<void ()> > >::_M_destroy_nodes(std::function<void ()>**, std::function<void ()>**) /usr/include/c++/9/bits/stl_deque.h:761
    #5 0x7f6efcb65f94 in std::_Deque_base<std::function<void ()>, std::allocator<std::function<void ()> > >::~_Deque_base() /usr/include/c++/9/bits/stl_deque.h:683
    #6 0x7f6efcb65f94 in std::deque<std::function<void ()>, std::allocator<std::function<void ()> > >::~deque() /usr/include/c++/9/bits/stl_deque.h:1072
    #7 0x7f6efcb65f94 in std::queue<std::function<void ()>, std::deque<std::function<void ()>, std::allocator<std::function<void ()> > > >::~queue() /usr/include/c++/9/bits/stl_queue.h:96
    #8 0x7f6efcb65f94 in dht::ThreadPool::~ThreadPool() /foo/contrib/native/opendht/src/thread_pool.cpp:55
    #9 0x7f6efb01d8a6  (/lib/x86_64-linux-gnu/libc.so.6+0x468a6)

previously allocated by thread T0 here:
    #0 0x7f6efe805587 in operator new(unsigned long) ../../../../src/libsanitizer/asan/asan_new_delete.cc:104
    #1 0x7f6efcb66979 in __gnu_cxx::new_allocator<std::function<void ()> >::allocate(unsigned long, void const*) /usr/include/c++/9/ext/new_allocator.h:114
    #2 0x7f6efcb66979 in std::allocator_traits<std::allocator<std::function<void ()> > >::allocate(std::allocator<std::function<void ()> >&, unsigned long) /usr/include/c++/9/bits/alloc_traits.h:443
    #3 0x7f6efcb66979 in std::_Deque_base<std::function<void ()>, std::allocator<std::function<void ()> > >::_M_allocate_node() /usr/include/c++/9/bits/stl_deque.h:620
    #4 0x7f6efcb66979 in std::_Deque_base<std::function<void ()>, std::allocator<std::function<void ()> > >::_M_create_nodes(std::function<void ()>**, std::function<void ()>**) /usr/include/c++/9/bits/stl_deque.h:745
    #5 0x7f6efcb66979 in std::_Deque_base<std::function<void ()>, std::allocator<std::function<void ()> > >::_M_initialize_map(unsigned long) /usr/include/c++/9/bits/stl_deque.h:719
    #6 0x7f6efcb65b44 in std::_Deque_base<std::function<void ()>, std::allocator<std::function<void ()> > >::_Deque_base() /usr/include/c++/9/bits/stl_deque.h:510
    #7 0x7f6efcb65b44 in std::deque<std::function<void ()>, std::allocator<std::function<void ()> > >::deque() /usr/include/c++/9/bits/stl_deque.h:915
    #8 0x7f6efcb65b44 in std::queue<std::function<void ()>, std::deque<std::function<void ()>, std::allocator<std::function<void ()> > > >::queue<std::deque<std::function<void ()>, std::allocator<std::function<void ()> > >, void>() /usr/include/c++/9/bits/stl_queue.h:161
    #9 0x7f6efcb65b44 in dht::ThreadPool::stop() /foo/contrib/native/opendht/src/thread_pool.cpp:109
    #10 0x7f6efcb65e69 in dht::ThreadPool::join() /foo/contrib/native/opendht/src/thread_pool.cpp:116
    #11 0x7f6efcb65e69 in dht::ThreadPool::~ThreadPool() /foo/contrib/native/opendht/src/thread_pool.cpp:57
    #12 0x7f6efb01d8a6  (/lib/x86_64-linux-gnu/libc.so.6+0x468a6)

SUMMARY: AddressSanitizer: double-free ../../../../src/libsanitizer/asan/asan_new_delete.cc:165 in operator delete(void*)
==168118==ABORTING

added typebug label

changed title from daemon-agent: use after-free to daemon-agent: double free

changed the description

More infos:

ALSA lib conf.c:4732:(_snd_config_evaluate) function snd_func_refer returned error: No such file or directory
ALSA lib conf.c:5220:(snd_config_expand) Evaluate error: No such file or directory
ALSA lib pcm.c:2642:(snd_pcm_open_noupdate) Unknown PCM default
free(): double free detected in tcache 2

exit -> atexit -> DRing::fini -> ThreadPool::join

Seems the same problem we got with Logger & agent where ThreadPool got a static destructor which is probably ran before atexit.

Related to https://github.com/savoirfairelinux/opendht/commit/1a267418ddcf19c900fcc32c3b3a4883416d7090

Solutions:

• Got a non static ThreadPool, like the logger
• use empty()/pop()
• Replace with asio thread pools.

Note that empty/pop is not bullet proof. It's just a consequence that the internal pointer of the queue -- that is mallocated -- is probably never deferenced. Reading the structure itself does not generate an error because it's in the .data segment and is therefore valid for the entire duration of the program.

Until then, Dring::fini is not safe to be used in an atexit handler.

I close for now. I think avoiding atexit is ok for the agent. the proper fix for opendht would be to remove ThreadLoop and uses asio instead, but it's not planned in short/mid-term

closed