Best fit for Promiscuous

[andyl]

Just saw your RubyConf video - wow! I have a question about finding the best fit for Promiscuous. Not sure if this is the best venue - but here goes...

Looks to me like promiscuous generates one-way information flows from a front-end system to a read-only companion app. So a good fit would be a web-app that feeds a reporting system.

Does promiscuous support bi-directional updates? How would I architect a SOA with promiscuous where the companion app also needs to update the data?

Other related questions: how to enable shared-session across apps, how to integrate the UI across apps? Can you give pointers to good forums or information sources for these issues?

Thanks in advance...

[nviennot]

Hi Andy,

Does promiscuous support bi-directional updates? How would I architect a SOA with promiscuous where the companion app also needs to update the data?

Let me make a distinction on what bi-directional updates mean to me:

1. The first case is to decorate an already published model, for example, application A publishes a user model, and application B is doing some machine learning to add a field to the user model like spammy:true/false. Application C would subscribe to both A and B, and aggregate the fields from the two apps and merge them. If A==B, then we would have loops, so promiscuous would have to be smart and figure it out and do the right thing. In general, you would want an arbitrary long chain. e.g. A publishes user information, B is a facebook crawler and decorate the user model with its number of friends, C does machine learning for spammy users, and A subscribes to C.

2. The second case is to have inter-app data dependencies. For example A publishes a user model, and B is a facebook crawler, and publishes Friend models with a foreign key to the user model. C subscribe to both A and B, and never want to see a invalid foreign key.

Both are hard to do, which I want to address Q1/Q2 2014. 2) is especially hard.

Right now, you can do 2) but, you don't have any consistency. For example, A publisher a model User, and B publishes a model UserB which A subscribes to. To decorate a model like in 1), you could have a remote observer of UserB, and in the before_save callback, User.find(id).update_attributes(), and you would be done.
These are hacks that we do at crowdtap, but offer no consistency guarantees.

Other related questions: how to enable shared-session across apps, how to integrate the UI across apps? Can you give pointers to good forums or information sources for these issues?

Now we are talking about synchronous APIs, which is not what promiscuous do. We found that publishing the access_token as a field doesn't work so well, as things starts to break down if a subscriber is lagging behind.
There is also an option of storing the current_user_id in a cookie, which can be readable by all your apps by sharing the same secret key to sign the cookie.

To integrate the UI across apps, you could hit different endpoints that routes to different applications. Or you can use the model decoration hacks to provide data directly from one app. Or you can have one app doing a synchronous query to another of your backend application, and give an aggregate result.

I hope this answer your question :)
We should probably create a mailing list, but I like github issues :)

[andyl]

Hi Nicolas -

Thanks for taking the time to answer my questions. It is great to hear that you are thinking about bi-directional updates and are going to take a crack at it.

I knew that loops would be an issue. I hadn't thought far enough ahead to consider the situation where there may be a chain of publishers/subscribers. I guess that would make loop-detection more difficult.

For my situation, I'm thinking about a SOA arranged with a star topology - a master app in the center, doing bi-directional updates with a set of add-on apps.

My timeframe is Q1/Q2, so I'll keep an eye on your progress. Drop me a line if you're looking for feedback or 3rd party use cases for bi-directional updates.

[nviennot]

Great :)

I am definitely looking for feedback!

[nathanpalmer]

@nviennot What about this scenario of bi-directional updates? Seems similar to 1)

3. A publishes User object, B is doing some machine learning and updates the spammy:true/false directly on User and A subscribes to User to get the updates back.

In this case I'd assume the user object on both sides is both a subscriber and a publisher in the DSL. In my scenario one side is ALWAYS handling creation and both sides can update. I assume there would be some issues with change conflict in the case that spammy gets updated in both places (which you could handle via time.)

[nviennot]

I've implemented this feature in my fork.
There is no documentation but this test illustrate the DSL: https://github.com/nviennot/promiscuous/blob/master/spec/integration/multi_pub_spec.rb#L111-L150

Each app is responsible for a set of field. If you break that rule (many apps updating the same field) you might get into infinite update loops issues.

Sorry if I'm not being super helpful.

[nathanpalmer]

Ok, interesting. Any plans to merge that into master?

[nviennot]

I'm pretty sure it is, @kareemk is now the maintainer. He has a lot on his plate, so don't expect it in the near future.

[kareemk]

As there is a fairly easy workaround by using a different model to store the additional field I've slated this for 2.0. Tracked by #56.

[nathanpalmer]

Sounds great @kareemk