Skip to content

Commit

Permalink
add support for the first cucumber messages (#1641)
Browse files Browse the repository at this point in the history
* add sending testrunstarted and file sink (#1571)

* add cucumber messages MyGet feed

* add Cucumber.Messages dependency to project

* add ICucumberMessageSender and implementation

* add ICucumberMessageSink

* add date and time abstraction

* move clock into SpecFlow project

* add SendTestRunStarted method

* use IClock interface in CucumberMessageSender

* add ICucumberMessageFactory

* use ICucumberMessageFactory and ICucumberMessageSink in CucumberMessageSender

* add unit test for CucumberMessageFactory

* update Cucumber.Messages reference

* make DateTimeClock to UtcDateTimeClock

* add tests for CucumberMessageFactory and CucumberMessageSender

* extend BuildTestRunStartedMessage to use timeStamp

* remove unnecessary using

* register and inject CucumberMessageSender

* add dependency to Cucumber.Messages to nuspec

* update submodule

* update submodule

* add failing tests

* update Cucumber.Messages

* set CucumberImplementation in CucumberMessageFactory

* update Cucumber.Messages version

* add IBinaryFileAccessor

* add Result<T> types

* fix failing test

* add ProtobufFileSink and tests

* add ProtobufFileSinkOutput and tests

* refactor CucumberMessageSender configuration

* add test for environment variables

* add factories for configuration and sinks

* register new dependencies

* replace sink factory with simple injection

* remove new configuration

* add mutex for system-wide locking instead of lock

* remove now unnecessary tests

* add ExceptionFailure class

* use ExceptionFailure

* remove unnecessary Initialize

* move generics to Success only

* add and use ISuccess<T>

* throw exception on writing error

* Integrate TestCaseStarted message (#1580)

* add BuildTestCaseStartedMessage and tests

* use adapted Result types

* refactor CucumberMessageSender

* refactor CucumberMessageFactory

* add test for SendTestCaseStarted

* add call to TestExecutionEngine

* add PickleIdGenerator

* register PickleIdGenerator

* fix padding

* change return type to Guid

* add PickleIdStore

* refactor pickleId parameters to Guid

* remove whitespace

* change return type of GetPickleIdForScenario

* use IPickleIdStore in TestExecutionEngine

* add parameter in test

* remove usings

* inject dictionary

* add tests for PickleIdStore

* refactor Result models

* use improved Result models

* add and use PickleIdStoreDictionaryFactory

* let IFailure<out T> derive from IFailure

* reorder interfaces

* derive Success<T> from Success

* add description to Failure

* add WrappedFailure

* change parameter type to IFailure

* extract method from WriteMessage

* adapt exception text

* add cucumber as submodule

* add external feature files to specs

* fix order of target execution

* add step definitions

* update submodule

* write to fixed file instead of random

* add message wrapper to factory

* use Wrapper in code

* use WriteDelimitedTo instead of WriteTo

* add binding and driver code

* add validation of fields

* update submodule

* refactor test suite setup into driver

* use TestSuiteInitializationDriver for setting up start time

* remove usage of ProjectsDriver

* add EnsureAProjectIsCreated method

* update submodule

* extend failure messages

* fix _isProjectCreated for 0-feature projects

* add IEnvironmentWrapper

* fix assertion step

* use IEnvironmentWrapper

* refactor CucumberMessagesDriver

* add step skeletons for TestCaseStarted

* update submodule

* send override env values for TestCaseStarted

* remove whitespace

* only create a project if no one has been created

* add TestCaseStartedDriver

* add binding logic for TestCaseStarted

* refactor pickleId string conversion into method

* add and test BuildTestCaseFinishedMessage

* fix test

* add TestResultFactory

* update cucumber submodule

* adapt bindings to changes in feature file

* add ITestRunContext interface

* add IProtobufFileNameResolver

* move environment variable resolving to IEnvironmentWrapper

* fix not compiling tests

* extract SpecFlow.dll path search

* refactor ITestRunContext

* extend EnvironmentWrapper

* add and register TestRunContexts

* improve usage of Success and Failure

* add single quotes

* add BuildWrapperMessage overload for TestCaseFinished

* update submodule

* update submodule

* adapt TestResultFactory to build from ScenarioContext

* use ITestResultFactory in TestExecutionEngine

* refactor CucumberMessageSender

* add SendTestCaseFinished to ICucumberMessageSender

* adapt tests

* add generation of passing step bindings

* add bindings for TestCaseFinished

* fix typos

* propagate errors

* trace errors

* refactor CucumberMessageSender

* update submodule

* fix missing dependency

* remove implicit generation of bindings

* add binding and code for TestCaseFinished status passed

* add missing file

* update submodule

* add case to TestResultFactory

* add unit tests

* remove old pattern from gitignore

* refactor error message fetching

* refactor CucumberMessageSenderValueMockSource

* rename CucumberMessageSenderValueMockSource to FieldValueProvider

* update submodule

* add unit tests

* improve failure message in TestCaseFinishedDriver

* add sending pending test results

* change pending message fetching in TestExecutionEngine

* update submodule

* add missing binding

* update submodule

* adapt binding name

* add unit tests

* remove not needed method

* add message factory

* add registration in DI container

* test case finished for undefined scenarios (#1604)

* update submodule

* add step bindings

* add unit tests

* add ITestUndefinedMessageFactory

* refactor ITestResultFactory

* add case for undefined step

* update submodule

* add bindings and logic for new steps

* add assembly hooks for MSTest

* add assembly hooks for NUnit

* remove call to OnTestRunStart in bindings registry initialization

* add assembly hooks for xUnit

* reformat XUnitTestGeneratorProvider classes

* set Collection attribute on xUnit test classes

* add generation of missing step bindings

* add parsing of inline step definitions from features

* move inline step definition types into separate files

* move drivers from SpecFlow to TestProjectGenerator

* fix typo

* rename AssertionsDriver to MessageValidationDriver

* include cucumber messages features in code-behind generation on first build

* fix hooks

* remove failing test

* disable specflow+runner specific scenario

* update submodule
  • Loading branch information
SabotageAndi authored Jul 1, 2019
1 parent 6996f3f commit 1a6b161
Show file tree
Hide file tree
Showing 96 changed files with 2,953 additions and 152 deletions.
2 changes: 1 addition & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -340,7 +340,6 @@ obj/
[Rr]elease*/
_ReSharper*/
_NCrunch*/
[Tt]est[Rr]esult*
*.pidb
*.userprefs
*.resharper
Expand Down Expand Up @@ -382,6 +381,7 @@ GitExtensions.settings.backup
/Installer/NuGetPackages/SpecFlow.Tools.MsBuild.Generation/build/SpecFlow.Tools.MsBuild.Generation.props
/Tests/TechTalk.SpecFlow.MsBuildNetSdk.IntegrationTests/Features/dummy.feature.cs
*.feature.cs
/Tests/TechTalk.SpecFlow.Specs/Features/CucumberMessages

# Nerdbank.GitVersioning
Tests/TechTalk.SpecFlow.Specs/NuGetPackageVersion.cs
3 changes: 3 additions & 0 deletions .gitmodules
Original file line number Diff line number Diff line change
@@ -1,3 +1,6 @@
[submodule "ExternalRepositories/SpecFlow.TestProjectGenerator"]
path = ExternalRepositories/SpecFlow.TestProjectGenerator
url = https://github.com/techtalk/SpecFlow.TestProjectGenerator.git
[submodule "ExternalRepositories/cucumber"]
path = ExternalRepositories/cucumber
url = https://github.com/techtalk/cucumber.git
2 changes: 1 addition & 1 deletion ExternalRepositories/SpecFlow.TestProjectGenerator
1 change: 1 addition & 0 deletions ExternalRepositories/cucumber
Submodule cucumber added at c254bf
Original file line number Diff line number Diff line change
Expand Up @@ -11,9 +11,15 @@
<EmbedUntrackedSources>true</EmbedUntrackedSources>
<AllowedOutputExtensionsInPackageBuildOutputFolder>$(AllowedOutputExtensionsInPackageBuildOutputFolder);.pdb</AllowedOutputExtensionsInPackageBuildOutputFolder>
</PropertyGroup>
<ItemGroup>
<Compile Remove="build\MSTest.AssemblyHooks.cs" />
</ItemGroup>
<ItemGroup>
<None Include="build\MSTest.AssemblyHooks.cs" />
</ItemGroup>

<ItemGroup>
<PackageReference Include="Microsoft.SourceLink.GitHub" Version="1.0.0-beta-63127-02" PrivateAssets="All"/>
<PackageReference Include="Microsoft.SourceLink.GitHub" Version="1.0.0-beta-63127-02" PrivateAssets="All" />
</ItemGroup>

<ItemGroup>
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
using System.Diagnostics;
using global::Microsoft.VisualStudio.TestTools.UnitTesting;
using global::TechTalk.SpecFlow;

[TestClass]
public class MSTestAssemblyHooks
{
[AssemblyInitialize]
public static void AssemblyInitialize(TestContext testContext)
{
var currentAssembly = typeof(MSTestAssemblyHooks).Assembly;

TestRunnerManager.OnTestRunStart(currentAssembly);
}

[AssemblyCleanup]
public static void AssemblyCleanup()
{
var currentAssembly = typeof(MSTestAssemblyHooks).Assembly;

TestRunnerManager.OnTestRunEnd(currentAssembly);
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
Imports Microsoft.VisualStudio.TestTools.UnitTesting
Imports TechTalk.SpecFlow
Imports System
Imports System.Reflection


<TestClass>
Public NotInheritable Class MSTestAssemblyHooks
<AssemblyInitialize>
Public Shared Sub AssemblyInitialize(testContext As TestContext)

Dim currentAssembly As Assembly = GetType(MSTestAssemblyHooks).Assembly

TestRunnerManager.OnTestRunStart(currentAssembly)
End Sub

<AssemblyCleanup>
Public Shared Sub AssemblyCleanup()

Dim currentAssembly As Assembly = GetType(MSTestAssemblyHooks).Assembly

TestRunnerManager.OnTestRunEnd(currentAssembly)
End Sub

End Class
Original file line number Diff line number Diff line change
@@ -1,5 +1,20 @@
<Project ToolsVersion="12.0" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">

<PropertyGroup>
<BuildDependsOn>
GenerateSpecFlowAssemblyHooksFileTask;
$(BuildDependsOn)
</BuildDependsOn>
<CleanDependsOn>
$(CleanDependsOn)
</CleanDependsOn>
<RebuildDependsOn>
GenerateSpecFlowAssemblyHooksFileTask;
$(RebuildDependsOn)
</RebuildDependsOn>
</PropertyGroup>


<PropertyGroup>

<_SpecFlow_MsTestGeneratorPlugin Condition=" '$(MSBuildRuntimeType)' == 'Core'">netstandard2.0</_SpecFlow_MsTestGeneratorPlugin>
Expand All @@ -10,8 +25,16 @@
<_SpecFlow_MsTestRuntimePlugin Condition=" '$(TargetFrameworkIdentifier)' == '.NETFramework' ">net45</_SpecFlow_MsTestRuntimePlugin>
<_SpecFlow_MsTestRuntimePluginPath>$(MSBuildThisFileDirectory)\..\lib\$(_SpecFlow_MsTestRuntimePlugin)\TechTalk.SpecFlow.MSTest.SpecFlowPlugin.dll</_SpecFlow_MsTestRuntimePluginPath>

<GeneratedSpecFlowAssemblyHooksFile Condition="'$(GeneratedSpecFlowAssemblyHooksFile)' == ''">$(MSBuildThisFileDirectory)MSTest.AssemblyHooks$(DefaultLanguageSourceExtension)</GeneratedSpecFlowAssemblyHooksFile>
<GenerateSpecFlowAssemblyHooksFile Condition="'$(GenerateSpecFlowAssemblyHooksFile)' == ''">true</GenerateSpecFlowAssemblyHooksFile>

</PropertyGroup>

<Target Name="GenerateSpecFlowAssemblyHooksFileTask" Condition="'$(GenerateSpecFlowAssemblyHooksFile)' == 'true'">
<ItemGroup Condition="'$(Language)' == 'VB' or '$(Language)' == 'C#'">
<Compile Include="$(GeneratedSpecFlowAssemblyHooksFile)"/>
</ItemGroup>
</Target>



</Project>
Original file line number Diff line number Diff line change
Expand Up @@ -11,9 +11,15 @@
<EmbedUntrackedSources>true</EmbedUntrackedSources>
<AllowedOutputExtensionsInPackageBuildOutputFolder>$(AllowedOutputExtensionsInPackageBuildOutputFolder);.pdb</AllowedOutputExtensionsInPackageBuildOutputFolder>
</PropertyGroup>
<ItemGroup>
<Compile Remove="build\NUnit.AssemblyHooks.cs" />
</ItemGroup>
<ItemGroup>
<None Include="build\NUnit.AssemblyHooks.cs" />
</ItemGroup>

<ItemGroup>
<PackageReference Include="Microsoft.SourceLink.GitHub" Version="1.0.0-beta-63127-02" PrivateAssets="All"/>
<PackageReference Include="Microsoft.SourceLink.GitHub" Version="1.0.0-beta-63127-02" PrivateAssets="All" />
</ItemGroup>

<ItemGroup>
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
using System.Diagnostics;
using global::NUnit.Framework;
using global::TechTalk.SpecFlow;

[SetUpFixture]
public class NUnitAssemblyHooks
{
[OneTimeSetUp]
public void AssemblyInitialize()
{
var currentAssembly = typeof(NUnitAssemblyHooks).Assembly;

TestRunnerManager.OnTestRunStart(currentAssembly);
}

[OneTimeTearDown]
public void AssemblyCleanup()
{
var currentAssembly = typeof(NUnitAssemblyHooks).Assembly;

TestRunnerManager.OnTestRunEnd(currentAssembly);
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
Imports NUnit.Framework
Imports TechTalk.SpecFlow
Imports System
Imports System.Reflection

<SetUpFixture>
Public NotInheritable Class NUnitAssemblyHooks
<OneTimeSetUp>
Public Shared Sub AssemblyInitialize()
Dim currentAssembly As Assembly = GetType(NUnitAssemblyHooks).Assembly

TestRunnerManager.OnTestRunStart(currentAssembly)
End Sub

<OneTimeTearDown>
Public Shared Sub AssemblyCleanup()
Dim currentAssembly As Assembly = GetType(NUnitAssemblyHooks).Assembly

TestRunnerManager.OnTestRunEnd(currentAssembly)
End Sub

End Class
Original file line number Diff line number Diff line change
@@ -1,5 +1,20 @@
<Project ToolsVersion="12.0" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">

<PropertyGroup>
<BuildDependsOn>
GenerateSpecFlowAssemblyHooksFileTask;
$(BuildDependsOn)
</BuildDependsOn>
<CleanDependsOn>
$(CleanDependsOn)
</CleanDependsOn>
<RebuildDependsOn>
GenerateSpecFlowAssemblyHooksFileTask;
$(RebuildDependsOn)
</RebuildDependsOn>
</PropertyGroup>


<PropertyGroup>

<_SpecFlow_NUnitGeneratorPlugin Condition=" '$(MSBuildRuntimeType)' == 'Core'">netstandard2.0</_SpecFlow_NUnitGeneratorPlugin>
Expand All @@ -10,8 +25,16 @@
<_SpecFlow_NUnitRuntimePlugin Condition=" '$(TargetFrameworkIdentifier)' == '.NETFramework' ">net45</_SpecFlow_NUnitRuntimePlugin>
<_SpecFlow_NUnitRuntimePluginPath>$(MSBuildThisFileDirectory)\..\lib\$(_SpecFlow_NUnitRuntimePlugin)\TechTalk.SpecFlow.NUnit.SpecFlowPlugin.dll</_SpecFlow_NUnitRuntimePluginPath>

<GeneratedSpecFlowAssemblyHooksFile Condition="'$(GeneratedSpecFlowAssemblyHooksFile)' == ''">$(MSBuildThisFileDirectory)NUnit.AssemblyHooks$(DefaultLanguageSourceExtension)</GeneratedSpecFlowAssemblyHooksFile>
<GenerateSpecFlowAssemblyHooksFile Condition="'$(GenerateSpecFlowAssemblyHooksFile)' == ''">true</GenerateSpecFlowAssemblyHooksFile>

</PropertyGroup>

<Target Name="GenerateSpecFlowAssemblyHooksFileTask" Condition="'$(GenerateSpecFlowAssemblyHooksFile)' == 'true'">
<ItemGroup Condition="'$(Language)' == 'VB' or '$(Language)' == 'C#'">
<Compile Include="$(GeneratedSpecFlowAssemblyHooksFile)"/>
</ItemGroup>
</Target>



</Project>
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
using System.IO;
using TechTalk.SpecFlow.Plugins;
using TechTalk.SpecFlow.TestFramework;

namespace TechTalk.SpecFlow.NUnit.SpecFlowPlugin
{
public class NUnitNetFrameworkTestRunContext : ITestRunContext
{
private readonly ISpecFlowPath _specFlowPath;

public NUnitNetFrameworkTestRunContext(ISpecFlowPath specFlowPath)
{
_specFlowPath = specFlowPath;
}

public string GetTestDirectory() => Path.GetDirectoryName(_specFlowPath.GetPathToSpecFlowDll());
}
}
10 changes: 9 additions & 1 deletion Plugins/TechTalk.SpecFlow.NUnit.SpecFlowPlugin/RuntimePlugin.cs
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
using TechTalk.SpecFlow.Infrastructure;
using TechTalk.SpecFlow.NUnit.SpecFlowPlugin;
using TechTalk.SpecFlow.Plugins;
using TechTalk.SpecFlow.TestFramework;
using TechTalk.SpecFlow.Tracing;
using TechTalk.SpecFlow.UnitTestProvider;


[assembly: RuntimePlugin(typeof(RuntimePlugin))]

namespace TechTalk.SpecFlow.NUnit.SpecFlowPlugin
Expand All @@ -13,10 +13,18 @@ public class RuntimePlugin : IRuntimePlugin
{
public void Initialize(RuntimePluginEvents runtimePluginEvents, RuntimePluginParameters runtimePluginParameters, UnitTestProviderConfiguration unitTestProviderConfiguration)
{
runtimePluginEvents.CustomizeGlobalDependencies += RuntimePluginEvents_CustomizeGlobalDependencies;
runtimePluginEvents.CustomizeScenarioDependencies += RuntimePluginEvents_CustomizeScenarioDependencies;
unitTestProviderConfiguration.UseUnitTestProvider("nunit");
}

private void RuntimePluginEvents_CustomizeGlobalDependencies(object sender, CustomizeGlobalDependenciesEventArgs e)
{
#if NETFRAMEWORK
e.ObjectContainer.RegisterTypeAs<NUnitNetFrameworkTestRunContext, ITestRunContext>();
#endif
}

private void RuntimePluginEvents_CustomizeScenarioDependencies(object sender, CustomizeScenarioDependenciesEventArgs e)
{
var container = e.ObjectContainer;
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -11,9 +11,15 @@
<EmbedUntrackedSources>true</EmbedUntrackedSources>
<AllowedOutputExtensionsInPackageBuildOutputFolder>$(AllowedOutputExtensionsInPackageBuildOutputFolder);.pdb</AllowedOutputExtensionsInPackageBuildOutputFolder>
</PropertyGroup>
<ItemGroup>
<Compile Remove="build\xUnit.AssemblyHooks.cs" />
</ItemGroup>
<ItemGroup>
<None Include="build\xUnit.AssemblyHooks.cs" />
</ItemGroup>

<ItemGroup>
<PackageReference Include="Microsoft.SourceLink.GitHub" Version="1.0.0-beta-63127-02" PrivateAssets="All"/>
<PackageReference Include="Microsoft.SourceLink.GitHub" Version="1.0.0-beta-63127-02" PrivateAssets="All" />
</ItemGroup>

<ItemGroup>
Expand Down
Original file line number Diff line number Diff line change
@@ -1,5 +1,19 @@
<Project ToolsVersion="12.0" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">

<PropertyGroup>
<BuildDependsOn>
GenerateSpecFlowAssemblyHooksFileTask;
$(BuildDependsOn)
</BuildDependsOn>
<CleanDependsOn>
$(CleanDependsOn)
</CleanDependsOn>
<RebuildDependsOn>
GenerateSpecFlowAssemblyHooksFileTask;
$(RebuildDependsOn)
</RebuildDependsOn>
</PropertyGroup>

<PropertyGroup>

<_SpecFlow_xUnitGeneratorPlugin Condition=" '$(MSBuildRuntimeType)' == 'Core'" >netstandard2.0</_SpecFlow_xUnitGeneratorPlugin>
Expand All @@ -10,8 +24,16 @@
<_SpecFlow_xUnitRuntimePlugin Condition=" '$(TargetFrameworkIdentifier)' == '.NETFramework' ">net45</_SpecFlow_xUnitRuntimePlugin>
<_SpecFlow_xUnitRuntimePluginPath>$(MSBuildThisFileDirectory)\..\lib\$(_SpecFlow_xUnitRuntimePlugin)\TechTalk.SpecFlow.xUnit.SpecFlowPlugin.dll</_SpecFlow_xUnitRuntimePluginPath>

<GeneratedSpecFlowAssemblyHooksFile Condition="'$(GeneratedSpecFlowAssemblyHooksFile)' == ''">$(MSBuildThisFileDirectory)xUnit.AssemblyHooks$(DefaultLanguageSourceExtension)</GeneratedSpecFlowAssemblyHooksFile>
<GenerateSpecFlowAssemblyHooksFile Condition="'$(GenerateSpecFlowAssemblyHooksFile)' == ''">true</GenerateSpecFlowAssemblyHooksFile>

</PropertyGroup>

<Target Name="GenerateSpecFlowAssemblyHooksFileTask" Condition="'$(GenerateSpecFlowAssemblyHooksFile)' == 'true'">
<ItemGroup Condition="'$(Language)' == 'VB' or '$(Language)' == 'C#'">
<Compile Include="$(GeneratedSpecFlowAssemblyHooksFile)"/>
</ItemGroup>
</Target>


</Project>
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
using global::System;
using global::Xunit;
using global::TechTalk.SpecFlow;

namespace InternalSpecFlow
{
public class XUnitAssemblyFixture
{
static XUnitAssemblyFixture()
{
var currentAssembly = typeof(XUnitAssemblyFixture).Assembly;

TestRunnerManager.OnTestRunStart(currentAssembly);
}
}
}

Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
Namespace InternalSpecFlow

Public Class XUnitAssemblyFixture

Shared Sub New()
Dim currentAssembly As System.Reflection.Assembly = GetType(XUnitAssemblyFixture).Assembly

Global.TechTalk.SpecFlow.TestRunnerManager.OnTestRunStart(currentAssembly)
End Sub

End Class

End Namespace
Loading

0 comments on commit 1a6b161

Please sign in to comment.