Idea: Use vftable installation to decide if an object is a base class or embedded

[sei-eschwartz]

Here is a godbolt showing a class inherited and embedded: https://godbolt.org/z/jK7hTc51b

The embedded instance is constructed after the vbtables are installed:

Outer::Outer(void) PROC ; Outer::Outer, COMDAT
  push esi
  mov esi, ecx
  call InheritMe::InheritMe(void) ; InheritMe::InheritMe
  lea ecx, DWORD PTR [esi+16]
  mov DWORD PTR [esi], OFFSET const Outer::`vftable'
  call EmbedMe::EmbedMe(void) ; EmbedMe::EmbedMe
  mov eax, esi
  mov BYTE PTR bool volatile outer, 1 ; outer
  pop esi
  ret 0
Outer::Outer(void) ENDP ; Outer::Outer

Can we use this to decide which instances are inheritance or embedding? I think the main confusion is inlining. Could we get confused if the outer class does not have any vftables, and the first embedded class does?

Here's a pathological example: https://godbolt.org/z/8ca7WP6nG

Outer::Outer(void) PROC ; Outer::Outer, COMDAT
  push esi
  mov esi, ecx
  mov BYTE PTR bool volatile base, 1 ; base
  mov BYTE PTR bool volatile inherit, 1 ; inherit
  mov BYTE PTR bool volatile base, 1 ; base
  mov DWORD PTR [esi+12], OFFSET const EmbedMe::`vftable'
  lea ecx, DWORD PTR [esi+28]
  mov BYTE PTR bool volatile embed, 1 ; embed
  call EmbedMe2::EmbedMe2(void) ; EmbedMe2::EmbedMe2
  mov eax, esi
  mov BYTE PTR bool volatile outer, 1 ; outer
  pop esi
  ret 0
Outer::Outer(void) ENDP ; Outer::Outer

Unfortunately, this example shows some larger problems with our rules. reasonVFTableBelongsToClass would actually conclude that EmbedMe::vftable belongs to Outer, which is incorrect.

I suppose my conclusion for now is that the idea of using vftables to distinguish inheritance or not is probably okay, since the only counterexamples I can think of would cause other major problems with our current rules.

[edmcman]

I'm thinking of rules we could add for this.

Perhaps a sanity check that all factDerivedClass appear before factEmbeddedObject? That seems true for regular inheritance. Is it true for virtual inheritance as well?

Placing embedded instances of the non-virtually inherited bases first,
• Adding a hidden vbptr unless a suitable one was inherited from one of the non-virtual
bases,
• Placing the new data members declared in the derived class, and, finally,
• Placing a single instance of each of the virtually inherited bases at the end of the
instance.

No :-( So we really need a way to represent virtual inheritance in order to implement this rule.