Skip to content
Snippets Groups Projects
New look: Off

Daemon blocked on IceTransport::send during shutdown

    • Open Issue created by François-Simon Fauteux-Chapleau @ffauteux-chapleau

    During shutdown, the daemon waits for all threads in the thread pools to join:

    [...]
#3  0x00007910d469e7a3 in __pthread_clockjoin_ex (threadid=133110991685312, thread_return=0x0, clockid=0, abstime=0x0, block=<optimized out>) at ./nptl/pthread_join_common.c:102
#4  0x00007910d4aece33 in std::thread::join() () at /lib/x86_64-linux-gnu/libstdc++.so.6
#5  0x00007910d5b4794d in dht::ThreadPool::join() () at /home/ffauteux-chapleau/jami-client-qt/daemon/src/.libs/libjami.so.0
#6  0x00007910d5355d42 in jami::Manager::finish (this=0x7910d6e149c0 <jami::Manager::instance()::instance>) at manager.cpp:916
#7  0x00007910d53e8593 in libjami::fini () at ring_api.cpp:100
[...]

    In the backtrace above, the thread with ID 133110991685312 is itself waiting for another thread to join:

    #0  0x00007910d4698d71 in __futex_abstimed_wait_common64 (private=128, cancel=true, abstime=0x0, op=265, expected=30217, futex_word=0x790f16200990) at ./nptl/futex-internal.c:57
#1  __futex_abstimed_wait_common (cancel=true, private=128, abstime=0x0, clockid=0, expected=30217, futex_word=0x790f16200990) at ./nptl/futex-internal.c:87
#2  __GI___futex_abstimed_wait_cancelable64 (futex_word=futex_word@entry=0x790f16200990, expected=30217, clockid=clockid@entry=0, abstime=abstime@entry=0x0, private=private@entry=128)
    at ./nptl/futex-internal.c:139
#3  0x00007910d469e7a3 in __pthread_clockjoin_ex (threadid=133105702667968, thread_return=0x0, clockid=0, abstime=0x0, block=<optimized out>) at ./nptl/pthread_join_common.c:102
#4  0x00007910d4aece33 in std::thread::join() () at /lib/x86_64-linux-gnu/libstdc++.so.6
#5  0x00007910d5a107eb in dhtnet::tls::TlsSession::TlsSessionImpl::~TlsSessionImpl() () at /home/ffauteux-chapleau/jami-client-qt/daemon/src/.libs/libjami.so.0
#6  0x00007910d5a10c54 in dhtnet::tls::TlsSession::~TlsSession() () at /home/ffauteux-chapleau/jami-client-qt/daemon/src/.libs/libjami.so.0
#7  0x00007910d5a10c7d in dhtnet::tls::TlsSession::~TlsSession() () at /home/ffauteux-chapleau/jami-client-qt/daemon/src/.libs/libjami.so.0
#8  0x00007910d59f6d6c in dhtnet::TlsSocketEndpoint::~TlsSocketEndpoint() () at /home/ffauteux-chapleau/jami-client-qt/daemon/src/.libs/libjami.so.0
#9  0x00007910d59f6e0d in dhtnet::TlsSocketEndpoint::~TlsSocketEndpoint() () at /home/ffauteux-chapleau/jami-client-qt/daemon/src/.libs/libjami.so.0
#10 0x00007910d59d5e48 in std::_Sp_counted_ptr_inplace<dhtnet::ConnectionInfo, std::allocator<void>, (__gnu_cxx::_Lock_policy)2>::_M_dispose() ()
    at /home/ffauteux-chapleau/jami-client-qt/daemon/src/.libs/libjami.so.0
#11 0x00007910d531266d in std::_Sp_counted_base<(__gnu_cxx::_Lock_policy)2>::_M_release_last_use (this=0x791044094970) at /usr/include/c++/13/bits/shared_ptr_base.h:175
#12 0x00007910d5309036 in std::_Sp_counted_base<(__gnu_cxx::_Lock_policy)2>::_M_release_last_use_cold (this=0x791044094970) at /usr/include/c++/13/bits/shared_ptr_base.h:199
#13 0x00007910d530387c in std::_Sp_counted_base<(__gnu_cxx::_Lock_policy)2>::_M_release (this=0x791044094970) at /usr/include/c++/13/bits/shared_ptr_base.h:353
#14 0x00007910d59b2e04 in std::_Function_handler<void (), dhtnet::ConnectionManager::Impl::connectDeviceOnNegoDone(std::weak_ptr<dhtnet::DeviceInfo> const&, std::shared_ptr<dhtnet::ConnectionInfo> const&, dht::Hash<32ul> const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, unsigned long const&, std::shared_ptr<dht::crypto::Certificate> const&)::{lambda(bool)#1}::operator()(bool) const::{lambda()#1}>::_M_manager(std::_Any_data&, std::_Any_data const&, std::_Manager_operation) ()
    at /home/ffauteux-chapleau/jami-client-qt/daemon/src/.libs/libjami.so.0
#15 0x00007910d5b48f0b in std::thread::_State_impl<std::thread::_Invoker<std::tuple<dht::ThreadPool::run(std::function<void ()>&&)::{lambda()#1}> > >::_M_run() ()
    at /home/ffauteux-chapleau/jami-client-qt/daemon/src/.libs/libjami.so.0
#16 0x00007910d4aecdb4 in ??? () at /lib/x86_64-linux-gnu/libstdc++.so.6
#17 0x00007910d469caa4 in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:447
#18 0x00007910d4729c3c in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:78

    However, this latter thread sometimes gets stuck waiting forever on a condition variable in IceTransport::send:

    #0  0x00007910d4698d71 in __futex_abstimed_wait_common64 (private=30991, cancel=true, abstime=0x0, op=393, expected=0, futex_word=0x791008375f98) at ./nptl/futex-internal.c:57
#1  __futex_abstimed_wait_common (cancel=true, private=30991, abstime=0x0, clockid=0, expected=0, futex_word=0x791008375f98) at ./nptl/futex-internal.c:87
#2  __GI___futex_abstimed_wait_cancelable64
    (futex_word=futex_word@entry=0x791008375f98, expected=expected@entry=0, clockid=clockid@entry=0, abstime=abstime@entry=0x0, private=private@entry=0) at ./nptl/futex-internal.c:139
#3  0x00007910d469b7ed in __pthread_cond_wait_common (abstime=0x0, clockid=0, mutex=0x791008375f48, cond=0x791008375f70) at ./nptl/pthread_cond_wait.c:503
#4  ___pthread_cond_wait (cond=0x791008375f70, mutex=0x791008375f48) at ./nptl/pthread_cond_wait.c:627
#5  0x00007910d59ddf99 in dhtnet::IceTransport::send(unsigned int, unsigned char const*, unsigned long) () at /home/ffauteux-chapleau/jami-client-qt/daemon/src/.libs/libjami.so.0
#6  0x00007910d59f6f6b in dhtnet::IceSocketEndpoint::write(unsigned char const*, unsigned long, std::error_code&) ()
    at /home/ffauteux-chapleau/jami-client-qt/daemon/src/.libs/libjami.so.0
#7  0x00007910d5a0cc6d in dhtnet::tls::TlsSession::TlsSessionImpl::sendRaw(void const*, unsigned long) () at /home/ffauteux-chapleau/jami-client-qt/daemon/src/.libs/libjami.so.0
#8  0x00007910d5a0cddb in dhtnet::tls::TlsSession::TlsSessionImpl::sendRawVec(iovec const*, int) () at /home/ffauteux-chapleau/jami-client-qt/daemon/src/.libs/libjami.so.0
#9  0x00007910d443d9e8 in ??? () at /lib/x86_64-linux-gnu/libgnutls.so.30
#10 0x00007910d443f410 in ??? () at /lib/x86_64-linux-gnu/libgnutls.so.30
#11 0x00007910d44729f5 in gnutls_alert_send () at /lib/x86_64-linux-gnu/libgnutls.so.30
#12 0x00007910d444985b in gnutls_bye () at /lib/x86_64-linux-gnu/libgnutls.so.30
#13 0x00007910d5a07cc1 in dhtnet::tls::TlsSession::TlsSessionImpl::cleanup() () at /home/ffauteux-chapleau/jami-client-qt/daemon/src/.libs/libjami.so.0
#14 0x00007910d5a29ef5 in dhtnet::ThreadLoop::mainloop(std::thread::id&, std::function<bool ()>, std::function<void ()>, std::function<void ()>) ()
    at /home/ffauteux-chapleau/jami-client-qt/daemon/src/.libs/libjami.so.0
#15 0x00007910d5a2a0da in std::thread::_State_impl<std::thread::_Invoker<std::tuple<void (dhtnet::ThreadLoop::*)(std::thread::id&, std::function<bool ()>, std::function<void ()>, std::function<void ()>), dhtnet::ThreadLoop*, std::reference_wrapper<std::thread::id>, std::function<bool ()>, std::function<void ()>, std::function<void ()> > > >::_M_run() ()
    at /home/ffauteux-chapleau/jami-client-qt/daemon/src/.libs/libjami.so.0
#16 0x00007910d4aecdb4 in ??? () at /lib/x86_64-linux-gnu/libstdc++.so.6
#17 0x00007910d469caa4 in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:447
#18 0x00007910d4729c3c in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:78

    Designs

      Child items ...

      2. Activity

      Please register or sign in to reply