Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ensure let blocks are evaluated only once per example #493

Closed
wants to merge 14 commits into from
Closed

Ensure let blocks are evaluated only once per example #493

wants to merge 14 commits into from

Conversation

sharplet
Copy link
Contributor

@sharplet sharplet commented Apr 3, 2014

This resolves #433.

  1. Instead of holding a direct reference to its block, KWLetNode now turns it into a future (using https://github.com/mikeash/MAFuture).

  2. When a node is evaluated, its pointer and all its children's pointers are assigned a reference to the future. The important difference here is that the block still hasn't been evaluated. Because it's now a future, the block isn't evaluated until the first time the future object is messaged, at which point the block is invoked and the message send is proxied to the real object.

    That is, we now have real lazy evaluation, just like RSpec! Yay! 🎉

  3. The let node tree is now unlinked, and the values reset, at the end of each example (rather than after the last example in a context, which was incorrect).

I'm still not 100% certain that this was the only solution. But figuring out the correct order in which to evaluate let nodes, in order to "simulate" lazy evaluation, was a big headache. Introducing futures made this problem go away by decoupling let node tree evaluation (assigning object references to let helper variables) from actual let block evaluation. So when the tree is evaluated we can just

EVALUATE ALL THE THINGS

without worrying about side effects. And then when a future is actually used the block is lazily evaluated as needed.

Hope that makes sense! 😅

And here's the functional test that verifies the behaviour: https://github.com/sharplet/Kiwi/blob/let-futures/Tests/KWFunctionalTests.m#L207-226.


If everyone's happy to go ahead with this, the question of the best way to import a third-party dependency still remains. I've just copied the source code from MAFuture HEAD into the tree for now, for testing. For CocoaPods, I've exported the code as a subspec. Would a git submodule be better for bringing in the source? Additionally, it's also bringing in a significant chunk of code that's still MRC, which doesn't make me super happy given the awesome progress recently on converting to ARC!

@yas375
Copy link
Contributor

yas375 commented Apr 10, 2014

I just want to say that this solves the problem I've got when I tried to use let.

@sharplet thanks a lot! I've upgraded to Kiwi 2.2.4 and have tried to rewrite some tests to use let, but have failed. Then I've tried to use your branch and everything works as supposed to! Thanks.

Here is how I'm using let:

describe(@"Foo", ^{
  __block NSDate *date;

  let(bars, ^id{
    // some code which uses `date` variable and returns NSArray
    return @[]; // some code which returns NSArray
  });

  context(@"when AAA", ^{
    it(@"returns aaa", ^{
      date = [NSDate mt_dateFromYear:2014 month:6 day:10];

      [[bars should] haveCountOf:1];
      [[bars should] containObjects:obj1, nil];
    });
  });

  context(@"when BBB", ^{
    it(@"returns bbb", ^{
      date = [NSDate mt_dateFromYear:2014 month:1 day:10];

      [[bars should] haveAtLeast:3];
      [[bars should] containObjects:obj2, nil];
    });
  });
});

Looking forward to see this in master and to have a new release :)

@yas375
Copy link
Contributor

yas375 commented Apr 10, 2014

Also a short question: when I add a breakpoint inside a block passed to let macros it stops a few lines below then supposed to. Is there an easy fix for this? Does it need a separate issue? Thanks.

2014-04-10_1216

@sharplet
Copy link
Contributor Author

sharplet commented May 4, 2014

@yas375 I believe this may just be the way it works with macros: see @dmeehan1968's comment here.

@supermarin
Copy link

@sharplet thank you very much for your work here.

I think we should merge this to fix the existing let bug, and then clean it up / potentially decouple from MAFutures later.

The problem occurs if someone already includes them in the codebase, it won't compile anymore.
@modocache thoughts?

@sharplet
Copy link
Contributor Author

@supermarin No problem. A couple of strategies I've thought about for MAFutures:

  1. Wrap the whole lot in a #ifdef macro, to enable users to compile it out if they have their own implementation (and I think there's a potential compile time vs. run time issue here, because if Kiwi is compiled into the test target and then injected into an app bundle, we could have some undefined behaviour).
  2. Create a podspec for MAFutures, include it as a submodule, and potentially contribute any updates we make (e.g., ARC) back upstream. It may be a good idea to maintain a fork of MAFutures under the kiwi-bdd org for this reason.
  3. Re-namespace everything: MAFutures -> KWFutures

I'm leaning towards 2 at the moment (or something close to it), on the grounds that contributing what we can back to the community is a good thing.

@supermarin
Copy link

/ping @sharplet

I'm thinking of releasing 2.3.
Not sure if we should get this fixed before shipping or leave the fix for a patch. Any thoughts?

Conflicts:
	Kiwi.podspec
	Kiwi.xcodeproj/project.pbxproj
	Tests/KWFunctionalTests.m
@sharplet
Copy link
Contributor Author

Yo.

I think we should merge it. I was considering merging MAFuture as a subtree of Kiwi, but decided I didn't really want to have all of MAFuture's history as part of Kiwi (for now).

I'm a little concerned about the compatibility issues, however. I almost think it would be better if they were likely to cause hard compiler errors, as that would be easier to debug. However because we'll be compiled in the user's test bundle and then injected into their app during testing, any possible issues are more likely to manifest as subtle runtime bugs.

Because the code is just copied in and shipped with Kiwi, we're really responsible for it. So I think I've convinced myself that we should re-namespace the code so we can be more certain about what will happen at runtime. Do you agree?

This basically treats MAFuture as an internal implementation detail of
Kiwi.

If an app using Kiwi for testing itself includes MAFuture, there likely
won't be duplicate symbol errors at compile time as Kiwi will be
compiled into the test bundle, and then injected at runtime. Prefixing
MAFuture symbols with KW_ mitigates the risk of undefined behaviour in
this scenario.
@supermarin
Copy link

Sounds good to me, since this is only the test target. @modocache final thoughts?

we're close to 🚢 !!! 🍻

@modocache
Copy link
Member

@supermarin Thanks for the ping!

I was considering merging MAFuture as a subtree of Kiwi, but decided I didn't really want to have all of MAFuture's history as part of Kiwi (for now).

I think that's a very wise decision.

Personally, this pull request is a bit too complex for my tastes, however. I'm not looking forward to maintaining the futures code. I'm also of the opinion that let blocks are "nice to haves", but not essential. Sorry if that sounds negative--just my two cents.

But considering let is already in Kiwi, and a lot of users seem to enjoy using it, the practical thing to do is to get it working! The functional test in the PR makes it very clear what problem this is solving, and overall I think the diff looks good, so I'm +1 for merging.

@supermarin
Copy link

@modocache thanks for the input!

My thoughts were the same - there must be a way to deal with it in a simpler way, I just haven't had time to dig too deep in this code anymore :(

@sharplet
Copy link
Contributor Author

sharplet commented Jul 1, 2014

@supermarin @modocache Definitely not negative, I agree this has gotten much more complicated. Pretty sure if I'd originally submitted a PR this heavy you would have politely declined ;) Thanks for the feedback!

sharplet added 3 commits July 8, 2014 18:41
Conflicts:
	Kiwi.xcodeproj/project.pbxproj
…ectations

When futures are involved, it seems that -isEqual: is not necessarily
commutative. For example, these two aren't guaranteed to be equivalent:

    [[futureObject should] equal:actualObject];
    [[actualObject should] equal:futureObject];

In the case of string comparison, it is the second form that fails. An
MAFuture is very lightweight proxy for the resolved future value --
almost all messages the future receives are forwarded to the real object
it represents. However when a future string is on the right-hand side of
-isEqual:, the comparison fails. My best guess is that the
implementation of -isEqualToString: doesn't send any messages to the
right operand, and accesses its internal storage directly.
@sharplet
Copy link
Contributor Author

sharplet commented Jul 8, 2014

@modocache @supermarin I've just added a failing test for an issue I discovered when using future strings in expectations, see the build results here: https://travis-ci.org/kiwi-bdd/Kiwi/builds/29403601.

I also thought of a potential fix and pushed it here: https://github.com/sharplet/Kiwi/commit/b57b4de786e1c5380df62dfabc8716c1616c455d.

I'm a little torn. The fix works great, and even though it does introduce a performance penalty in the negative case, I feel like the logic is valid. Still, I can't quite fight the feeling that maybe it's too much of a special case to fix it with such a sledgehammer—and of course the underlying problem still exists.

Of course, if you have any ideas for an alternate fix I'm all ears! 😉

@orta
Copy link
Contributor

orta commented May 21, 2016

This looks like an interesting experiment, and I've just discussed this with @sharplet and @modocache and the complexity it adds probably isn't worth it. So, as it's been inactive for over two years, I'm closing this.

As ever - this PR can be re-opened if the idea is worth bringing back up, but it looks like discussion has petered out.

@orta orta closed this May 21, 2016
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

let blocks in nested contexts are executed multiple times
5 participants