GtkD crash: 'BadAlloc (insufficient resources for operation)'

Permalink: HTTP NNTP

mark

Posted Tue, 28 Apr 2020 09:22:10 GMT

Reply

I've copied this over from the D Learn list because I haven't had a response so far.

I'm assuming the problem is with my code. Anyway the full source is here:
https://github.com/mark-summerfield/debfind

And here's the problem:

When I ran the program I did Find 'memoize' which worked. Then 'memoize python' which also worked. Then said to find 'any word' (which produces 1000s of rows) at which point it crashed. Below is the bt. Does it look like my bug or a Gtk or GtkD bug?

GNU gdb (Ubuntu 8.1-0ubuntu3.2) 8.1.0.20180409-git
[snip]
Reading symbols from DebFind...done.
(gdb) run
Starting program: /home/mark/app/d/debfind/DebFind
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
[New Thread 0x7fffeddfc700 (LWP 2441)]
[New Thread 0x7fffed5fb700 (LWP 2442)]
[New Thread 0x7fffe7fff700 (LWP 2443)]
[New Thread 0x7ffff7fe2700 (LWP 2444)]
[New Thread 0x7ffff7fdc700 (LWP 2445)]
[New Thread 0x7ffff7fd6700 (LWP 2446)]
[New Thread 0x7ffff7e64700 (LWP 2447)]
[New Thread 0x7ffff7e5e700 (LWP 2448)]
[New Thread 0x7ffff7e58700 (LWP 2449)]
[New Thread 0x7ffff7e52700 (LWP 2450)]
warning: Corrupted shared library list: 0x5555560a5e60 != 0x555555dabe00
[New Thread 0x7fffc20ff700 (LWP 2451)]
[New Thread 0x7fffc18fe700 (LWP 2452)]
[Thread 0x7fffe7fff700 (LWP 2443) exited]
[Thread 0x7fffc18fe700 (LWP 2452) exited]
populateNames
populateNames
populateNames

(DebFind:2436): Gdk-ERROR **: 15:48:40.080: The program 'DebFind' received an X Window System error.
This probably reflects a bug in the program.
The error was 'BadAlloc (insufficient resources for operation)'.
  (Details: serial 12790 error_code 11 request_code 130 (MIT-SHM) minor_code 5)
  (Note to programmers: normally, X errors are reported asynchronously;
   that is, you will receive the error a while after causing it.
   To debug your program, run it with the GDK_SYNCHRONIZE environment
   variable to change this behavior. You can then get a meaningful
   backtrace from your debugger if you break on the gdk_x_error() function.)

Thread 1 "DebFind" received signal SIGTRAP, Trace/breakpoint trap.
0x00007ffff66c3ea1 in ?? () from /usr/lib/x86_64-linux-gnu/libglib-2.0.so.0
(gdb) bt
#0  0x00007ffff66c3ea1 in  () at /usr/lib/x86_64-linux-gnu/libglib-2.0.so.0
#1  0x00007ffff66c6819 in g_log_writer_default () at /usr/lib/x86_64-linux-gnu/libglib-2.0.so.0
#2  0x00007ffff66c4a8e in g_log_structured_array () at /usr/lib/x86_64-linux-gnu/libglib-2.0.so.0
#3  0x00007ffff66c54ce in g_log_structured_standard () at /usr/lib/x86_64-linux-gnu/libglib-2.0.so.0
#4  0x00007ffff5f62c41 in  () at /usr/lib/x86_64-linux-gnu/libgdk-3.so.0
#5  0x00007ffff5f6fac3 in  () at /usr/lib/x86_64-linux-gnu/libgdk-3.so.0
warning: (Internal error: pc 0x55555579a2d4 in read in psymtab, but not in symtab.)

warning: (Internal error: pc 0x55555579a280 in read in psymtab, but not in symtab.)

warning: (Internal error: pc 0x55555579a2d4 in read in psymtab, but not in symtab.)

#6  0x00007ffff37138fa in _XError () at /usr/lib/x86_64-linux-gnu/libX11.so.6
warning: (Internal error: pc 0x55555579a2d4 in read in psymtab, but not in symtab.)

#7  0x00007ffff371082b in  () at /usr/lib/x86_64-linux-gnu/libX11.so.6
#8  0x00007ffff37108d5 in  () at /usr/lib/x86_64-linux-gnu/libX11.so.6
#9  0x00007ffff3711205 in _XEventsQueued () at /usr/lib/x86_64-linux-gnu/libX11.so.6
#10 0x00007ffff3702d3d in XPending () at /usr/lib/x86_64-linux-gnu/libX11.so.6
#11 0x00007ffff5f6a09e in  () at /usr/lib/x86_64-linux-gnu/libgdk-3.so.0
#12 0x00007ffff66bdb28 in g_main_context_prepare () at /usr/lib/x86_64-linux-gnu/libglib-2.0.so.0
#13 0x00007ffff66be4fb in  () at /usr/lib/x86_64-linux-gnu/libglib-2.0.so.0
warning: (Internal error: pc 0x55555579a2d4 in read in psymtab, but not in symtab.)

#14 0x00007ffff66be6dc in g_main_context_iteration () at /usr/lib/x86_64-linux-gnu/libglib-2.0.so.0
warning: (Internal error: pc 0x55555579a2d4 in read in psymtab, but not in symtab.)

#15 0x00007ffff537eefd in g_application_run () at /usr/lib/x86_64-linux-gnu/libgio-2.0.so.0
warning: (Internal error: pc 0x55555579a2d4 in read in psymtab, but not in symtab.)

#16 0x000055555579a2d5 in warning: (Internal error: pc 0x55555579a2d4 in read in psymtab, but not in symtab.)

_D3gio11ApplicationQn3runMFAAyaZiwarning: (Internal error: pc 0x55555579a2d4 in read in psymtab, but not in symtab.)

warning: (Internal error: pc 0x55555579a2d4 in read in psymtab, but not in symtab.)

 (warning: (Internal error: pc 0x55555579a2d4 in read in psymtab, but not in symtab.)

this=0x7ffff7ed0360, warning: (Internal error: pc 0x55555579a2d4 in read in psymtab, but not in symtab.)

argv=...)warning: (Internal error: pc 0x55555579a2d4 in read in psymtab, but not in symtab.)

 at Application.dwarning: (Internal error: pc 0x55555579a2d4 in read in psymtab, but not in symtab.)

warning: (Internal error: pc 0x55555579a2d4 in read in psymtab, but not in symtab.)

:931
#17 0x0000555555784380 in D main (args=...) at app.d:18

Re: GtkD crash: 'BadAlloc (insufficient resources for operation)'

Permalink: HTTP NNTP

Russel Winder

Posted Tue, 28 Apr 2020 11:12:28 GMT in reply to mark

Reply

Mark,

I cloned the repository, and built the code, but on execution I got:

Running ./DebFind 
failed to load config: will use defaults

(DebFind:95981): GdkPixbuf-WARNING **: 12:10:29.070: Inline XPM data is broken: Dimensions do not match data
glib.ConstructionException.ConstructionException@../../../../../.dub/packages/gtk-d-3.9.0/gtk-d/generated/gtkd/glib/ConstructionException.d(37): null returned by new_from_xpm_data
----------------
Pixbuf.d:679 [0x55d5002f49b3]
appwindow.d:60 [0x55d500290569]
app.d:16 [0x55d5002e0e78]
DClosure.d:122 [0x55d5004ccff3]
??:? g_closure_invoke [0x7f472bfb4fd1]
??:? [0x7f472bfc8783]
??:? g_signal_emit_valist [0x7f472bfd354e]
??:? g_signal_emit [0x7f472bfd3ede]
??:? [0x7f472bc9fe17]
??:? g_application_run [0x7f472bc9ffc9]
Application.d:931 [0x55d5002f8f34]
app.d:18 [0x55d5002e0dbf]
??:? void rt.dmain2._d_run_main2(char[][], ulong, extern (C) int function(char[][])*).runAll() [0x7f47386b0e7f]
??:? _d_run_main2 [0x7f47386b0c8e]
??:? _d_run_main [0x7f47386b0aed]
entrypoint.d:35 [0x55d5002e0ea4]
??:? __libc_start_main [0x7f47382c2e0a]
??:? [0x55d500253c29]
Program exited with code 1

which I suspect is nothing at all to do with the problem you were hoping we would take a look at.

Re: GtkD crash: 'BadAlloc (insufficient resources for operation)'

Permalink: HTTP NNTP

mark

Posted Tue, 28 Apr 2020 11:36:52 GMT in reply to Russel Winder

Reply

Russel thanks for looking at it. I've commented out the icon stuff which you rightly saw was irrelevant anyway so hopefully it will build and run and crash "properly" now!

Re: GtkD crash: 'BadAlloc (insufficient resources for operation)'

Permalink: HTTP NNTP

Russel Winder

Posted Tue, 28 Apr 2020 12:04:06 GMT in reply to mark

Reply

I seem not to be able to use the UI such that the find button becomes live. Everything else seems to do something.

Re: GtkD crash: 'BadAlloc (insufficient resources for operation)'

Permalink: HTTP NNTP

mark

Posted Tue, 28 Apr 2020 12:17:43 GMT in reply to Russel Winder

Reply

On Tue, 28 Apr 2020 12:04:06 GMT, Russel Winder wrote:

I seem not to be able to use the UI such that the find button becomes live. Everything else seems to do something.

It takes some seconds to create the cache reading 10s of MBs of Packages files... After that Find is enabled and you're good to go.

Re: GtkD crash: 'BadAlloc (insufficient resources for operation)'

Permalink: HTTP NNTP

Russel Winder

Posted Tue, 28 Apr 2020 14:02:30 GMT in reply to mark

Reply

I suspect something weird, I have had the application running for many minutes and still no change, it still seems to be "Reading package files…".

Re: GtkD crash: 'BadAlloc (insufficient resources for operation)'

Permalink: HTTP NNTP

mark

Posted Tue, 28 Apr 2020 14:52:04 GMT in reply to Russel Winder

Reply

On Tue, 28 Apr 2020 14:02:30 GMT, Russel Winder wrote:

I suspect something weird, I have had the application running for many minutes and still no change, it still seems to be "Reading package files…".

On my Ubuntu 18.04 LTS 64-bit it takes 1.5sec to read the Packages files and a further 4.1sec to index them (and create a ~20MB cache in /tmp).

The packages are read from /var/lib/apt/lists/Packages (the dir and glob are in model.d). That dir has ~300MB of Packages files on my machine. Maybe you have a lot more?

If you try deleting /tmp/debfind-2020-04-28.cache and then run 'dub test' you should just see the cache being recreated and the CLI tests running. Then if that works and you try running 'dub' it should run the DebFind GUI which should detect the cache and be ready straight away.

I can't help thinking that I'd have been better off writing this in wxPython...
I really like D, but don't like Gtk at all.

Re: GtkD crash: 'BadAlloc (insufficient resources for operation)'

Permalink: HTTP NNTP

Mike Wey

Posted Tue, 28 Apr 2020 23:48:20 +0200 in reply to mark

Reply

On 28-04-2020 16:52, mark wrote:

On Tue, 28 Apr 2020 14:02:30 GMT, Russel Winder wrote:

I suspect something weird, I have had the application running for many minutes and still no change, it still seems to be "Reading package files…".

On my Ubuntu 18.04 LTS 64-bit it takes 1.5sec to read the Packages files and a further 4.1sec to index them (and create a ~20MB cache in /tmp).

The packages are read from /var/lib/apt/lists/Packages (the dir and glob are in model.d). That dir has ~300MB of Packages files on my machine. Maybe you have a lot more?

If you try deleting /tmp/debfind-2020-04-28.cache and then run 'dub test' you should just see the cache being recreated and the CLI tests running. Then if that works and you try running 'dub' it should run the DebFind GUI which should detect the cache and be ready straight away.

I can't help thinking that I'd have been better off writing this in wxPython...
I really like D, but don't like Gtk at all.

I can reproduce the error, for me it finds ~60k packages and on first
try it will show ~35k of packages in the listview. If i then search for
something and than clear the search i get the error when it's rebuilding
the list of ~35k packages.

I am not sure whats causing the error, a google search would suggest
some issues with available graphics memory. The error message may be
correct and you are really running out of resources, AFAIK every
GtkWidget has an X11 window associated with it, with to columns every
row has 2 label widgets and the list of ~35k rows holds handles on ~70k
X11 windows and maybe more depending on how the rows are implemented in gtk.

Altough gtk doesn't document what the limit is for a liststore, you may
need to implement a custom tree/list model to display your data.

Re: GtkD crash: 'BadAlloc (insufficient resources for operation)'

Permalink: HTTP NNTP

Russel Winder

Posted Wed, 29 Apr 2020 08:48:10 GMT in reply to mark

Reply

On Tue, 28 Apr 2020 14:52:04 GMT, mark wrote:

On my Ubuntu 18.04 LTS 64-bit it takes 1.5sec to read the Packages files and a further 4.1sec to index them (and create a ~20MB cache in /tmp).

The packages are read from /var/lib/apt/lists/Packages (the dir and glob are in model.d). That dir has ~300MB of Packages files on my machine. Maybe you have a lot more?

|> ll  /var/lib/apt/lists/*Packages* 
-rw-r--r-- 1 root root   870491 Mar 21 23:05 /var/lib/apt/lists/dimen.winder.org.uk:9999_d-apt_dists_d-apt_main_binary-amd64_Packages
-rw-r--r-- 1 root root   256863 Apr 26 21:06 /var/lib/apt/lists/dimen.winder.org.uk:9999_debian_dists_unstable_contrib_binary-amd64_Packages
-rw-r--r-- 1 root root    27796 Apr 26 21:06 /var/lib/apt/lists/dimen.winder.org.uk:9999_debian_dists_unstable_contrib_binary-amd64_Packages.diff_Index
-rw-r--r-- 1 root root 46166225 Apr 28 21:11 /var/lib/apt/lists/dimen.winder.org.uk:9999_debian_dists_unstable_main_binary-amd64_Packages
-rw-r--r-- 1 root root    27910 Apr 28 21:11 /var/lib/apt/lists/dimen.winder.org.uk:9999_debian_dists_unstable_main_binary-amd64_Packages.diff_Index
-rw-r--r-- 1 root root   529594 Apr 28 09:06 /var/lib/apt/lists/dimen.winder.org.uk:9999_debian_dists_unstable_non-free_binary-amd64_Packages
-rw-r--r-- 1 root root    27796 Apr 28 09:06 /var/lib/apt/lists/dimen.winder.org.uk:9999_debian_dists_unstable_non-free_binary-amd64_Packages.diff_Index
-rw-r--r-- 1 root root     2208 Sep 10  2019 /var/lib/apt/lists/dimen.winder.org.uk:9999_rlw_._Packages
-rw-r--r-- 1 root root     5146 Apr 24 14:21 /var/lib/apt/lists/dimen.winder.org.uk:9999_vivaldi_dists_stable_main_binary-amd64_Packages
-rw-r--r-- 1 root root  1199029 Apr 29 07:12 /var/lib/apt/lists/dimen.winder.org.uk:9999_vscode_dists_stable_main_binary-amd64_Packages

I am using Debian Sid. But I use approx (which runs on port 9999) as I have four machines to update and do not really want to download everything four times – approx acts as a cache.

If you try deleting /tmp/debfind-2020-04-28.cache and then run 'dub test' you should just see the cache being recreated and the CLI tests running. Then if that works and you try running 'dub' it should run the DebFind GUI which should detect the cache and be ready straight away.

|> dub test
Generating test runner configuration 'debfind-test-application' for 'application' (executable).
Excluding main source file src/app.d from test.
Performing "unittest" build using /usr/bin/ldc2 for x86_64.
aaset 0.2.5: building configuration "library"...
gtk-d:gtkd 3.9.0: target for configuration "library" is up to date.
debfind ~master: building configuration "debfind-test-application"...
Linking...
To force a rebuild of up-to-date targets, run again with --force.
Running ./debfind-test-application 
stemmer.d unittests: checked 23,531 stems
Reading package files…
/usr/lib/ldc/x86_64-linux-gnu/include/d/std/conv.d(2312): [unittest] Unexpected 'N' when converting from type string to type ulong
std.conv.ConvException@/usr/lib/ldc/x86_64-linux-gnu/include/d/std/conv.d(2312): Unexpected 'N' when converting from type string to type ulong
----------------
conv.d:2312 [0x55d90cefbf41]
conv.d:1905 [0x55d90cefbc6c]
conv.d:222 [0x55d90ce7431c]
modelutil.d:115 [0x55d90ce7421b]
modelutil.d:80 [0x55d90ce73a56]
modelutil.d:44 [0x55d90ce7348b]
parallelism.d:1811 [0x55d90ceea1e2]
??:? void std.parallelism.TaskPool.doJob(std.parallelism.AbstractTask*) [0x7f7fdd29f8b0]
??:? void std.parallelism.TaskPool.startWorkLoop() [0x7f7fdd29fa5a]
??:? thread_entryPoint [0x7f7fdcf648cb]
??:? [0x7f7fdce9cf26]
??:? clone [0x7f7fdcc6d2ee]
1/4 unittests FAILED
Program exited with code 1

I can't help thinking that I'd have been better off writing this in wxPython...
I really like D, but don't like Gtk at all.

I am a fan of GTK+ compared to Qt, so Rust/gtk-rs and D/GtkD are my friends!

Re: GtkD crash: 'BadAlloc (insufficient resources for operation)'

Permalink: HTTP NNTP

mark

Posted Wed, 29 Apr 2020 09:10:21 GMT in reply to Mike Wey

Reply

I replaced my simple ListStore subclass with an implementation of the TreeModelIF (based on the code in the GtkD demos DemoCustomList example) and updated on github/debfind. And I get exactly the same crash.

If you've any other suggestions I'll have one last try, but otherwise I think I've reached the end of the road with GtkD!