Skip to content

Fix three uses of a stale address after GC compaction - #232

Open
jeremy wants to merge 3 commits into
xml4r:masterfrom
jeremy:gc-compaction-fixes
Open

Fix three uses of a stale address after GC compaction#232
jeremy wants to merge 3 commits into
xml4r:masterfrom
jeremy:gc-compaction-fixes

Conversation

@jeremy

@jeremy jeremy commented Aug 7, 2026

Copy link
Copy Markdown

Summary

This pull request fixes issue #231. Three parts of the extension keep an address
into memory that the garbage collector controls. GC compaction moves the target.
The extension then uses the old address.

One commit fixes one defect. Each commit adds a regression test.

# Defect Effect Commit
1 The pointer registry keeps raw wrapper addresses The garbage collector stops the process 6553637
2 libxml2 receives a raw VALUE as the read context Segmentation fault fc5225e
3 XML::Reader.string does not keep the String Parse errors on corrupt data 64c0aa8

Test environment

  • ruby 4.0.6 (2026-07-14 revision 03b6d3f889) +PRISM [arm64-darwin23]
  • libxml2 2.9.13
  • macOS on arm64

Each test fails on the code before the commit. Each test passes after the commit.
Each test result is the same in 3 runs.

1. The pointer registry keeps raw wrapper addresses

ruby_xml_registry.c maps an xmlDocPtr or an xmlNodePtr to its Ruby wrapper.
It keeps each wrapper as a raw machine address. GC compaction moves the wrapper,
but the address in the registry stays the same. The address is then dead.

Mark functions read the registry. rxml_node_mark, rxml_dtd_mark and
rxml_reader_mark send the address to rb_gc_mark. The garbage collector then
stops the process:

[BUG] try to mark T_NONE object (obj: 0x... T_NONE/, parent: 0x... LibXML::XML::Node)
[BUG] Segmentation fault at 0x000000000000001c

The report says that dcompact cannot repair the registry, because dcompact
receives the data pointer and not the VALUE. That statement is not correct.
The data pointer is the key of the registry entry. So each type repairs its
own entry.

The new function rxml_registry_update does this. It finds the entry, calls
rb_gc_location on the stored address, and stores the result.

Two types add entries to the registry, so two types get a dcompact function:
the Document type and the managed Node type.

2. libxml2 receives a raw VALUE as the read context

XML::Parser::Context.io, XML::HTMLParser::Context.io and XML::Reader.io
give libxml2 a raw VALUE as the read context. libxml2 keeps the context and
calls rxml_read_callback much later. GC compaction moves the IO object in that
time. The callback then uses a dead address:

[BUG] Segmentation fault at 0x0000000000000010

The public methods XML::Parser.io, XML::Document.io, XML::SaxParser.io,
XML::HTMLParser.io and XML::Reader.io all use one of these three methods.

XML::Writer already shows the correct pattern. It gives libxml2 a struct that
holds the VALUE, and it marks that VALUE from its dmark function.
rb_gc_mark pins the object, so the address in the struct stays correct.

This pull request adds rxml_io_context for the same pattern:

  • The two parser contexts keep the struct in ctxt->_private. libxml2 never
    uses that field.
  • An xmlTextReader is opaque. So XML::Reader now wraps a new
    rxml_reader_object struct. The struct holds the reader and the read context.

The commit also removes the @io instance variable from the three classes. The
mark function now keeps the IO object alive. Please note that in the two parser
contexts the instance variable never worked. The code assigned the result of
ID2SYM to an ID, and then passed it to rb_ivar_set. The result is that
XML::Parser::Context.io(io).instance_variables is empty.

3. XML::Reader.string does not keep the String

XML::Reader.string calls xmlReaderForMemory with the address of the buffer of
the Ruby String. xmlReaderForMemory does not copy the buffer. libxml2 reads
from the buffer on each call to XML::Reader#read. The reader does not keep the
String, so the garbage collector frees the String. The reader then reads free
memory:

LibXML::XML::Error: Fatal error: Couldn't find end of Start Tag roo<garbage> at :1.

The corrupt name is other data in the reused memory.

The reader now keeps a frozen copy of the String and marks it with rb_gc_mark.
A frozen copy shares the buffer of the original String. So a large document costs
no more memory. The copy also protects the reader if the program changes the
original String. rb_gc_mark pins the copy. So GC compaction cannot move a short
String that holds its bytes inside the object.

Test method

Each regression test keeps the object under test in an Array. A local variable
is not correct for this test. The machine stack scan pins a local variable, so an
object in a local variable never moves.

The test for XML::Reader.string builds the String at run time and then makes the
garbage collector reuse the memory. A short literal String stays in place, so a
test with a literal String passes for the wrong reason.

test/test_helper.rb gets a new compact_heap method. The method skips the test
if the ruby build does not support compaction.

Results

Point Result
ef4b8eb (before) 400 runs, 22236 assertions, 0 failures, 0 errors, 0 skips
after commit 1 402 runs, 22241 assertions, 0 failures, 0 errors, 0 skips
after commit 2 405 runs, 22271 assertions, 0 failures, 0 errors, 0 skips
after commit 3 406 runs, 22272 assertions, 0 failures, 0 errors, 0 skips

The suite also passes with RUBY_FREE_AT_EXIT=1.

Questions

  1. rxml_node_mark still uses rb_gc_mark for the document. That call pins the
    document. With the registry repaired, rb_gc_mark_movable is now correct and
    lets the document move. Do you want that change?
  2. The two parser contexts use ctxt->_private. Recent libxml2 versions make the
    structures opaque. Do you prefer a wrapper struct for these two classes also?
  3. Is the removal of the @io instance variable acceptable? No code in the
    repository reads it. Please note one difference between the three classes. In
    the two parser contexts the instance variable never existed, because of the
    ID2SYM defect above. In XML::Reader it did exist. So the removal is a
    visible change for XML::Reader only. I can keep the instance variable there
    if you prefer.

jeremy added 3 commits August 6, 2026 18:26
The bindings map an xmlDocPtr or an xmlNodePtr to its Ruby wrapper in a
global st_table. The table holds each wrapper as a raw machine address.
GC compaction moves the wrapper but does not change the address in the
table. The table then holds a dead address.

Mark functions read the table. rxml_node_mark and rxml_dtd_mark and
rxml_reader_mark send the address to rb_gc_mark. The garbage collector
then aborts the process:

    [BUG] try to mark T_NONE object (obj: 0x... T_NONE/,
          parent: 0x... LibXML::XML::Node)
    [BUG] Segmentation fault at 0x000000000000001c

The data pointer that dcompact receives is the key of the table entry.
So each wrapper type can repair its own entry. Add rxml_registry_update
for this. It looks up the entry, calls rb_gc_location on the stored
address, and stores the result.

Install a dcompact function on the Document type and on the managed Node
type. These are the two types that add entries to the table.

Add regression tests for both types. The tests keep the wrapper in an
Array, not in a local variable. The machine stack scan pins a local
variable, so a wrapper in a local variable never moves.
XML::Parser::Context.io and XML::HTMLParser::Context.io and XML::Reader.io
give libxml2 a raw VALUE as the read context. libxml2 keeps that value
and calls rxml_read_callback much later. GC compaction moves the IO
object in the meantime, so the callback reads a dead address:

    [BUG] Segmentation fault at 0x0000000000000010

The public methods XML::Parser.io, XML::Document.io, XML::SaxParser.io,
XML::HTMLParser.io and XML::Reader.io all use one of these three methods.

XML::Writer already shows the correct pattern. It gives libxml2 a struct
that holds the VALUE, and it marks that VALUE from its dmark function.
rb_gc_mark pins the object, so the address in the struct stays correct.

Add rxml_io_context for this pattern:

  - The two parser contexts store the struct in ctxt->_private. libxml2
    never touches that field. The mark function marks the io object and
    the free function releases the struct.

  - An xmlTextReader is opaque, so XML::Reader now wraps a new
    rxml_reader_object struct that holds the reader and the io context.

Remove the @io instance variable from all three classes. The mark
function now keeps the io object alive. In the two parser contexts the
variable never worked, because the code assigned the result of ID2SYM to
an ID and then passed it to rb_ivar_set.

Add regression tests for XML::Parser.io, XML::HTMLParser.io and
XML::Reader.io. Each test keeps the parser or the reader in an Array, not
in a local variable. The machine stack scan pins a local variable, so an
object in a local variable never moves.
XML::Reader.string calls xmlReaderForMemory with the address of the
buffer of the Ruby String. xmlReaderForMemory does not copy the buffer,
and libxml2 reads from the buffer on each call to XML::Reader#read. The
reader does not keep the String, so the garbage collector frees the
String and the reader then reads free memory:

    LibXML::XML::Error: Fatal error:
      Couldn't find end of Start Tag roo<garbage> at :1.

The corrupted name is other data in the reused memory.

The reader now keeps a frozen copy of the String and marks it with
rb_gc_mark. A frozen copy shares the buffer of the original String, so
this does not copy the data of a large document. The copy also protects
the reader if the program changes the original String. rb_gc_mark pins
the copy, so GC compaction cannot move a short String that holds its
bytes inside the object.

Both constructors now wrap the struct in the Ruby object before they
store a VALUE in it. TypedData_Wrap_Struct allocates, so it can start a
garbage collection. A VALUE that only malloc memory holds is not visible
to the garbage collector at that moment.

Add a regression test. The test builds the string at run time and makes
the garbage collector reuse the memory. A short literal string stays in
place, so a test with a literal passes for the wrong reason.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants