Skip to content

Commit

Permalink
Sync to upstream/release/603 (#1097)
Browse files Browse the repository at this point in the history
# What's changed?

- Record the location of properties for table types (closes #802)
- Implement stricter UTF-8 validations as per the RFC
(luau-lang/rfcs#1)
- Implement `buffer` as a new type in both the old and new solvers.
- Changed errors produced by some `buffer` builtins to be a bit more
generic to avoid platform-dependent error messages.
- Fixed a bug where `Unifier` would copy some persistent types, tripping
some internal assertions.
- Type checking rules on relational operators is now a little bit more
lax.
- Improve dead code elimination for some `if` statements with complex
always-false conditions

## New type solver

- Dataflow analysis now generates phi nodes on exit of branches.
- Dataflow analysis avoids producing a new definition for locals or
properties that are not owned by that loop.
- If a function parameter has been constrained to `never`, report errors
at all uses of that parameter within that function.
- Switch to using the new `Luau::Set` to replace `std::unordered_set` to
alleviate some poor allocation characteristics which was negatively
affecting overall performance.
- Subtyping can now report many failing reasons instead of just the
first one that we happened to find during the test.
- Subtyping now also report reasons for type pack mismatches.
- When visiting `if` statements or expressions, the resulting context
are the common terms in both branches.

## Native codegen

- Implement support for `buffer` builtins to its IR for x64 and A64.
- Optimized `table.insert` by not inserting a table barrier if it is
fastcalled with a constant.

## Internal Contributors

Co-authored-by: Aaron Weiss <[email protected]>
Co-authored-by: Alexander McCord <[email protected]>
Co-authored-by: Andy Friesen <[email protected]>
Co-authored-by: Arseny Kapoulkine <[email protected]>
Co-authored-by: Aviral Goel <[email protected]>
Co-authored-by: Lily Brown <[email protected]>
Co-authored-by: Vyacheslav Egorov <[email protected]>
  • Loading branch information
8 people authored Nov 10, 2023
1 parent 7105c81 commit c2ba105
Show file tree
Hide file tree
Showing 113 changed files with 3,604 additions and 1,080 deletions.
2 changes: 2 additions & 0 deletions Analysis/include/Luau/ConstraintGenerator.h
Original file line number Diff line number Diff line change
Expand Up @@ -148,6 +148,8 @@ struct ConstraintGenerator
*/
ScopePtr childScope(AstNode* node, const ScopePtr& parent);

std::optional<TypeId> lookup(Scope* scope, DefId def);

/**
* Adds a new constraint with no dependencies to a given scope.
* @param scope the scope to add the constraint to.
Expand Down
10 changes: 9 additions & 1 deletion Analysis/include/Luau/ConstraintSolver.h
Original file line number Diff line number Diff line change
Expand Up @@ -3,14 +3,18 @@
#pragma once

#include "Luau/Constraint.h"
#include "Luau/DenseHash.h"
#include "Luau/Error.h"
#include "Luau/Location.h"
#include "Luau/Module.h"
#include "Luau/Normalize.h"
#include "Luau/ToString.h"
#include "Luau/Type.h"
#include "Luau/TypeCheckLimits.h"
#include "Luau/TypeFwd.h"
#include "Luau/Variant.h"

#include <utility>
#include <vector>

namespace Luau
Expand Down Expand Up @@ -74,6 +78,10 @@ struct ConstraintSolver
std::unordered_map<BlockedConstraintId, std::vector<NotNull<const Constraint>>, HashBlockedConstraintId> blocked;
// Memoized instantiations of type aliases.
DenseHashMap<InstantiationSignature, TypeId, HashInstantiationSignature> instantiatedAliases{{}};
// Breadcrumbs for where a free type's upper bound was expanded. We use
// these to provide more helpful error messages when a free type is solved
// as never unexpectedly.
DenseHashMap<TypeId, std::vector<std::pair<Location, TypeId>>> upperBoundContributors{nullptr};

// A mapping from free types to the number of unresolved constraints that mention them.
DenseHashMap<TypeId, size_t> unresolvedConstraints{{}};
Expand Down Expand Up @@ -140,7 +148,7 @@ struct ConstraintSolver
std::pair<std::vector<TypeId>, std::optional<TypeId>> lookupTableProp(
TypeId subjectType, const std::string& propName, bool suppressSimplification = false);
std::pair<std::vector<TypeId>, std::optional<TypeId>> lookupTableProp(
TypeId subjectType, const std::string& propName, bool suppressSimplification, std::unordered_set<TypeId>& seen);
TypeId subjectType, const std::string& propName, bool suppressSimplification, DenseHashSet<TypeId>& seen);

void block(NotNull<const Constraint> target, NotNull<const Constraint> constraint);
/**
Expand Down
59 changes: 34 additions & 25 deletions Analysis/include/Luau/DataFlowGraph.h
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@

// Do not include LValue. It should never be used here.
#include "Luau/Ast.h"
#include "Luau/ControlFlow.h"
#include "Luau/DenseHash.h"
#include "Luau/Def.h"
#include "Luau/Symbol.h"
Expand Down Expand Up @@ -74,11 +75,18 @@ struct DataFlowGraph
struct DfgScope
{
DfgScope* parent;
bool isLoopScope;

DenseHashMap<Symbol, const Def*> bindings{Symbol{}};
DenseHashMap<const Def*, std::unordered_map<std::string, const Def*>> props{nullptr};

std::optional<DefId> lookup(Symbol symbol) const;
std::optional<DefId> lookup(DefId def, const std::string& key) const;

void inherit(const DfgScope* childScope);

bool canUpdateDefinition(Symbol symbol) const;
bool canUpdateDefinition(DefId def, const std::string& key) const;
};

struct DataFlowResult
Expand Down Expand Up @@ -106,31 +114,32 @@ struct DataFlowGraphBuilder

std::vector<std::unique_ptr<DfgScope>> scopes;

DfgScope* childScope(DfgScope* scope);

void visit(DfgScope* scope, AstStatBlock* b);
void visitBlockWithoutChildScope(DfgScope* scope, AstStatBlock* b);

void visit(DfgScope* scope, AstStat* s);
void visit(DfgScope* scope, AstStatIf* i);
void visit(DfgScope* scope, AstStatWhile* w);
void visit(DfgScope* scope, AstStatRepeat* r);
void visit(DfgScope* scope, AstStatBreak* b);
void visit(DfgScope* scope, AstStatContinue* c);
void visit(DfgScope* scope, AstStatReturn* r);
void visit(DfgScope* scope, AstStatExpr* e);
void visit(DfgScope* scope, AstStatLocal* l);
void visit(DfgScope* scope, AstStatFor* f);
void visit(DfgScope* scope, AstStatForIn* f);
void visit(DfgScope* scope, AstStatAssign* a);
void visit(DfgScope* scope, AstStatCompoundAssign* c);
void visit(DfgScope* scope, AstStatFunction* f);
void visit(DfgScope* scope, AstStatLocalFunction* l);
void visit(DfgScope* scope, AstStatTypeAlias* t);
void visit(DfgScope* scope, AstStatDeclareGlobal* d);
void visit(DfgScope* scope, AstStatDeclareFunction* d);
void visit(DfgScope* scope, AstStatDeclareClass* d);
void visit(DfgScope* scope, AstStatError* error);
DfgScope* childScope(DfgScope* scope, bool isLoopScope = false);
void join(DfgScope* parent, DfgScope* a, DfgScope* b);

ControlFlow visit(DfgScope* scope, AstStatBlock* b);
ControlFlow visitBlockWithoutChildScope(DfgScope* scope, AstStatBlock* b);

ControlFlow visit(DfgScope* scope, AstStat* s);
ControlFlow visit(DfgScope* scope, AstStatIf* i);
ControlFlow visit(DfgScope* scope, AstStatWhile* w);
ControlFlow visit(DfgScope* scope, AstStatRepeat* r);
ControlFlow visit(DfgScope* scope, AstStatBreak* b);
ControlFlow visit(DfgScope* scope, AstStatContinue* c);
ControlFlow visit(DfgScope* scope, AstStatReturn* r);
ControlFlow visit(DfgScope* scope, AstStatExpr* e);
ControlFlow visit(DfgScope* scope, AstStatLocal* l);
ControlFlow visit(DfgScope* scope, AstStatFor* f);
ControlFlow visit(DfgScope* scope, AstStatForIn* f);
ControlFlow visit(DfgScope* scope, AstStatAssign* a);
ControlFlow visit(DfgScope* scope, AstStatCompoundAssign* c);
ControlFlow visit(DfgScope* scope, AstStatFunction* f);
ControlFlow visit(DfgScope* scope, AstStatLocalFunction* l);
ControlFlow visit(DfgScope* scope, AstStatTypeAlias* t);
ControlFlow visit(DfgScope* scope, AstStatDeclareGlobal* d);
ControlFlow visit(DfgScope* scope, AstStatDeclareFunction* d);
ControlFlow visit(DfgScope* scope, AstStatDeclareClass* d);
ControlFlow visit(DfgScope* scope, AstStatError* error);

DataFlowResult visitExpr(DfgScope* scope, AstExpr* e);
DataFlowResult visitExpr(DfgScope* scope, AstExprGroup* group);
Expand Down
3 changes: 1 addition & 2 deletions Analysis/include/Luau/Def.h
Original file line number Diff line number Diff line change
Expand Up @@ -79,8 +79,7 @@ struct DefArena
TypedAllocator<Def> allocator;

DefId freshCell(bool subscripted = false);
// TODO: implement once we have cases where we need to merge in definitions
// DefId phi(const std::vector<DefId>& defs);
DefId phi(DefId a, DefId b);
};

} // namespace Luau
1 change: 1 addition & 0 deletions Analysis/include/Luau/Error.h
Original file line number Diff line number Diff line change
Expand Up @@ -322,6 +322,7 @@ struct TypePackMismatch
{
TypePackId wantedTp;
TypePackId givenTp;
std::string reason;

bool operator==(const TypePackMismatch& rhs) const;
};
Expand Down
4 changes: 2 additions & 2 deletions Analysis/include/Luau/Frontend.h
Original file line number Diff line number Diff line change
Expand Up @@ -71,7 +71,7 @@ struct SourceNode

ModuleName name;
std::string humanReadableName;
std::unordered_set<ModuleName> requireSet;
DenseHashSet<ModuleName> requireSet{{}};
std::vector<std::pair<ModuleName, Location>> requireLocations;
bool dirtySourceModule = true;
bool dirtyModule = true;
Expand Down Expand Up @@ -206,7 +206,7 @@ struct Frontend
std::vector<ModuleName>& buildQueue, const ModuleName& root, bool forAutocomplete, std::function<bool(const ModuleName&)> canSkip = {});

void addBuildQueueItems(std::vector<BuildQueueItem>& items, std::vector<ModuleName>& buildQueue, bool cycleDetected,
std::unordered_set<Luau::ModuleName>& seen, const FrontendOptions& frontendOptions);
DenseHashSet<Luau::ModuleName>& seen, const FrontendOptions& frontendOptions);
void checkBuildQueueItem(BuildQueueItem& item);
void checkBuildQueueItems(std::vector<BuildQueueItem>& items);
void recordItemResult(const BuildQueueItem& item);
Expand Down
2 changes: 2 additions & 0 deletions Analysis/include/Luau/Module.h
Original file line number Diff line number Diff line change
Expand Up @@ -102,6 +102,8 @@ struct Module
DenseHashMap<const AstType*, TypeId> astResolvedTypes{nullptr};
DenseHashMap<const AstTypePack*, TypePackId> astResolvedTypePacks{nullptr};

DenseHashMap<TypeId, std::vector<std::pair<Location, TypeId>>> upperBoundContributors{nullptr};

// Map AST nodes to the scope they create. Cannot be NotNull<Scope> because
// we need a sentinel value for the map.
DenseHashMap<const AstNode*, Scope*> astScopes{nullptr};
Expand Down
17 changes: 11 additions & 6 deletions Analysis/include/Luau/Normalize.h
Original file line number Diff line number Diff line change
Expand Up @@ -2,14 +2,14 @@
#pragma once

#include "Luau/NotNull.h"
#include "Luau/Set.h"
#include "Luau/TypeFwd.h"
#include "Luau/UnifierSharedState.h"

#include <initializer_list>
#include <map>
#include <memory>
#include <unordered_map>
#include <unordered_set>
#include <vector>

namespace Luau
Expand Down Expand Up @@ -254,6 +254,10 @@ struct NormalizedType
// This type is either never or thread.
TypeId threads;

// The buffer part of the type.
// This type is either never or buffer.
TypeId buffers;

// The (meta)table part of the type.
// Each element of this set is a (meta)table type, or the top `table` type.
// An empty set denotes never.
Expand Down Expand Up @@ -299,6 +303,7 @@ struct NormalizedType
bool hasNumbers() const;
bool hasStrings() const;
bool hasThreads() const;
bool hasBuffers() const;
bool hasTables() const;
bool hasFunctions() const;
bool hasTyvars() const;
Expand Down Expand Up @@ -359,7 +364,7 @@ class Normalizer
void unionTablesWithTable(TypeIds& heres, TypeId there);
void unionTables(TypeIds& heres, const TypeIds& theres);
bool unionNormals(NormalizedType& here, const NormalizedType& there, int ignoreSmallerTyvars = -1);
bool unionNormalWithTy(NormalizedType& here, TypeId there, std::unordered_set<TypeId>& seenSetTypes, int ignoreSmallerTyvars = -1);
bool unionNormalWithTy(NormalizedType& here, TypeId there, Set<TypeId>& seenSetTypes, int ignoreSmallerTyvars = -1);

// ------- Negations
std::optional<NormalizedType> negateNormal(const NormalizedType& here);
Expand All @@ -381,15 +386,15 @@ class Normalizer
std::optional<TypeId> intersectionOfFunctions(TypeId here, TypeId there);
void intersectFunctionsWithFunction(NormalizedFunctionType& heress, TypeId there);
void intersectFunctions(NormalizedFunctionType& heress, const NormalizedFunctionType& theress);
bool intersectTyvarsWithTy(NormalizedTyvars& here, TypeId there, std::unordered_set<TypeId>& seenSetTypes);
bool intersectTyvarsWithTy(NormalizedTyvars& here, TypeId there, Set<TypeId>& seenSetTypes);
bool intersectNormals(NormalizedType& here, const NormalizedType& there, int ignoreSmallerTyvars = -1);
bool intersectNormalWithTy(NormalizedType& here, TypeId there, std::unordered_set<TypeId>& seenSetTypes);
bool intersectNormalWithTy(NormalizedType& here, TypeId there, Set<TypeId>& seenSetTypes);
bool normalizeIntersections(const std::vector<TypeId>& intersections, NormalizedType& outType);

// Check for inhabitance
bool isInhabited(TypeId ty);
bool isInhabited(TypeId ty, std::unordered_set<TypeId> seen);
bool isInhabited(const NormalizedType* norm, std::unordered_set<TypeId> seen = {});
bool isInhabited(TypeId ty, Set<TypeId> seen);
bool isInhabited(const NormalizedType* norm, Set<TypeId> seen = {nullptr});

// Check for intersections being inhabited
bool isIntersectionInhabited(TypeId left, TypeId right);
Expand Down
2 changes: 1 addition & 1 deletion Analysis/include/Luau/Scope.h
Original file line number Diff line number Diff line change
Expand Up @@ -56,7 +56,6 @@ struct Scope
void addBuiltinTypeBinding(const Name& name, const TypeFun& tyFun);

std::optional<TypeId> lookup(Symbol sym) const;
std::optional<TypeId> lookupLValue(DefId def) const;
std::optional<TypeId> lookup(DefId def) const;
std::optional<std::pair<TypeId, Scope*>> lookupEx(DefId def);
std::optional<std::pair<Binding*, Scope*>> lookupEx(Symbol sym);
Expand All @@ -80,6 +79,7 @@ struct Scope
// types here.
DenseHashMap<const Def*, TypeId> rvalueRefinements{nullptr};

void inheritAssignments(const ScopePtr& childScope);
void inheritRefinements(const ScopePtr& childScope);

// For mutually recursive type aliases, it's important that
Expand Down
105 changes: 105 additions & 0 deletions Analysis/include/Luau/Set.h
Original file line number Diff line number Diff line change
@@ -0,0 +1,105 @@
// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details
#pragma once

#include "Luau/DenseHash.h"

namespace Luau
{

template<typename T>
using SetHashDefault = std::conditional_t<std::is_pointer_v<T>, DenseHashPointer, std::hash<T>>;

// This is an implementation of `unordered_set` using `DenseHashMap<T, bool>` to support erasure.
// This lets us work around `DenseHashSet` limitations and get a more traditional set interface.
template<typename T, typename Hash = SetHashDefault<T>>
class Set
{
private:
DenseHashMap<T, bool, Hash> mapping;
size_t entryCount = 0;

public:
Set(const T& empty_key)
: mapping{empty_key}
{
}

bool insert(const T& element)
{
bool& entry = mapping[element];
bool fresh = !entry;

if (fresh)
{
entry = true;
entryCount++;
}

return fresh;
}

template<class Iterator>
void insert(Iterator begin, Iterator end)
{
for (Iterator it = begin; it != end; ++it)
insert(*it);
}

void erase(const T& element)
{
bool& entry = mapping[element];

if (entry)
{
entry = false;
entryCount--;
}
}

void clear()
{
mapping.clear();
entryCount = 0;
}

size_t size() const
{
return entryCount;
}

bool empty() const
{
return entryCount == 0;
}

size_t count(const T& element) const
{
const bool* entry = mapping.find(element);
return (entry && *entry) ? 1 : 0;
}

bool contains(const T& element) const
{
return count(element) != 0;
}

bool operator==(const Set<T>& there) const
{
// if the sets are unequal sizes, then they cannot possibly be equal.
if (size() != there.size())
return false;

// otherwise, we'll need to check that every element we have here is in `there`.
for (auto [elem, present] : mapping)
{
// if it's not, we'll return `false`
if (present && there.contains(elem))
return false;
}

// otherwise, we've proven the two equal!
return true;
}
};

} // namespace Luau
5 changes: 2 additions & 3 deletions Analysis/include/Luau/Simplify.h
Original file line number Diff line number Diff line change
Expand Up @@ -2,11 +2,10 @@

#pragma once

#include "Luau/DenseHash.h"
#include "Luau/NotNull.h"
#include "Luau/TypeFwd.h"

#include <set>

namespace Luau
{

Expand All @@ -16,7 +15,7 @@ struct SimplifyResult
{
TypeId result;

std::set<TypeId> blockedTypes;
DenseHashSet<TypeId> blockedTypes;
};

SimplifyResult simplifyIntersection(NotNull<BuiltinTypes> builtinTypes, NotNull<TypeArena> arena, TypeId ty, TypeId discriminant);
Expand Down
Loading

0 comments on commit c2ba105

Please sign in to comment.