The problem#

Using my Factorio reflection library discussed previously involves interacting with Lua values that are a combination of the actual value and some metadata, so you have to know about those values to use them. Worse, the interactions I defined are quite verbose. The main thing you're like to want to do on a value is lookup a property on it. Normally in Lua that looks like

table[key]

but if instead of table you have a wrapped value from the reflection library, you would look up key on it with

ReflectionLibraryMod.typed_object_lookup_property(
    wrapped, key).value

If you want to do multiple levels of property lookups, then this quickly gets quite unwieldy.

The solution#

Lua supports operator overloading through a mechanism it calls metatables (some additional examples).

Using that mechanism, the library defines a value ReflectionLibraryMod.typed_data_raw that can be indexed as wrapped[key] and assigned to like wrapped[key] = newValue.

The basic setup looks like

local prototype = {} -- table for methods
local mt = {}
mt.__index = function (table, key)
  local res = prototype[key]
    or ReflectionLibraryMod.wrap_typed_object(
      ReflectionLibraryMod.typed_object_lookup_property(
        table._private, key))
  if res == nil then
    if key == "_value" then
      res = table._private.value
    end
  end
  return res
end

mt.__newindex = function (table, key, newValue)
  -- If newValue is a wrapped typed value, then unwrap it.
  if getmetatable(newValue) == mt then
    newValue = newValue._private.value
  end
  table._private.value[key] = newValue
end

function ReflectionLibraryMod.wrap_typed_object(typedValue)
  if typedValue == nil then
    return nil
  end

  local res = {_private = typedValue}
  setmetatable(res, mt)

  return res
end

Any additional properties would be defined next to the definition of _value. And any methods would be defined on prototype.

The details#

Where to store the real data#

Since Lua only lets us intercept table indexing and assignments on keys that are not present in the table, the only real member of the table is _private. Clients could read _private, but the name makes it clear they're not supposed to. Another way of handling that would be to declare the metatable inside a function such that _private was a variable local to the function, sometimes called the closure approach. There's some disagreement on which way of doing things is better; I figured it's not worth trying to do something complicated to protect the data, I just wanted to limit the keys that could conflict.

Using a closure would look something like this:

function ReflectionLibraryMod.wrap_typed_object(typedValue)
  if typedValue == nil then
    return nil
  end

  -- Define metatable inside this function
  --  so typedValue is in scope.
  local prototype = {}
  local mt = {}

  function mt.__index(table, key)
    return prototype[key]
      or ReflectionLibraryMod.wrap_typed_object(
        ReflectionLibraryMod.typed_object_lookup_property(
          typedValue, key))
  end

  setmetatable(res, mt)

  return res
end

Handling iteration#

The other thing that we need to fake to make our table look like the real one is pairs()/ipairs(). Metatables can control the behavior of those functions, allowing us to enhance the illusion. The code is only a slight modification of this example implementation that works like the default behavior, just iterating over table._private.value instead of table.

Debugger display#

The Factorio modding tools debugger separately supports a custom display of values with some additional metatable functions: __debugline for the string to display and __debugcontents for an iterator over the properties to display. That way, values with metatables can show more (or less) of their information in the debugger or just organize their properties differently. ReflectionLibrary uses this to display the type information in the debugger even though pairs() won't find it when iterating the object.

Declaring methods#

As defined above, the prototype table contains the methods available on any object using the metatable. Methods defined like

prototype._pathString = function (tbl)
  return ReflectionLibraryMod.typed_value_name(tbl._private)
end

can be called using typed:_pathString() (note the : which passes the table the lookup is done on as the first argument to the function).

Handling nil#

One issue that remains is that when indexing into the table, if the value is not present, the return value should be nil or at least count as a false boolean value. But we also want to be able to somehow access the type information on nil properties, at least if they're nil because it's an optional value that's not specified as opposed to being an invalid property name.

In the first iteration, this requires checking ._value == nil which is unidiomatic and awkward. A better, more idiomatic solution (suggested by justarandomgeek, the author of the Factorio Modding ToolKit) would be to use Lua's multiple results feature. The first result could be actually nil for nil lookups while the second result would always be a wrapped value using our metatable, so the type information could be looked up on it.

I have not implemented this change yet, but the basic idea would be to change the mt.__index method to something like this:

mt.__index = function (table, key)
  local res = prototype[key]
  if res ~= nil then
    return res
  end
  res = ReflectionLibraryMod.wrap_typed_object(
      ReflectionLibraryMod.typed_object_lookup_property(
        table._private, key))
  if res == nil then
    if key == "_value" then
      res = table._private.value
    end
    return res
  else
    local firstRes = res
    if res._private.value == nil then
      firstRes = nil
    end
    return firstRes, res
  end
end

Then if wrapped[key] then ... works as you'd expect from a real table, but also you can do _, valueWrapped = wrapped[key] to definitely get a wrapped value, even if it's wrapping nil.