Is this recipe still maintained?

[tisto]

I am wondering if this recipe is still maintained and used by people. @do3cc @saily?

At kitconcept, we moved to writing the Solr configuration via XML and we deploy Solr via Ansible scripts. This makes it a lot easier to debug and adapt to new Solr versions. Though, a buildout recipe is, of course, more convenient for Plone devs. Opinions?

[saily]

@tisto fully agree!

After spending a lot of time fighting with this recipe i ended up no longer upgrading solr because it just was too painful. If i would have to use Solr with Plone nowadays i would spin up a solr container for this.

[do3cc]

+1. This recipe is another example where adding another layer just increases complexity. Von meinem iPhone gesendet Am 15.05.2017 um 22:02 schrieb Daniel Widerin <[email protected]<mailto:[email protected]>>: @tisto<https://github.com/tisto> fully agree! After spending a lot of time fighting with this recipe i ended up no longer upgrading solr because it just was too painful. If i would have to use Solr with Plone nowadays i would spin up a solr container for this. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub<#62 (comment)>, or mute the thread<https://github.com/notifications/unsubscribe-auth/AAJAOMSqNusVm4VEbqXtQtf50j1kAWytks5r6K9lgaJpZM4NblS2>.

[mauritsvanrees]

I use the recipe (version 6.0.0b3) for one client. It helped me get started, so that was good. But the last release was over two years ago. Several useful fixes have been done, a few by me, but no one seems willing to release another version.
I can imagine that, because the master branch, though useful, is limited to by now old Solr versions. It is hard to keep the recipe up to date with the upstream developments.

If this recipe is abandoned, it would be great if people would share how they have set up Solr for Plone, and put some basic configuration in documentation somewhere, probably in collective.solr.

cc @fredvd

[gforcada]

I'm in the same position/opinion as @mauritsvanrees without this recipe it would have probably taken far more time to get started with Solr and while I did notice some shortcomings from the very beginning it made the transition quick and relatively easy.

If we want to ditch it for another approach, let's share that approach and document it (or give pointers on where to look for that) so that new Plone developers that want to start using Solr are not left out cold, specially given that some Plone knowledge is needed to configure Solr properly.

[fredvd]

@gforcada coming back to this recipe to update another site project. Yes we are still using this recipe for a few sites, that's also why Maurits did some updates last year.

It's not only the Plone knowledge that's needed for solr (that's mainly encoded in collective.solr), this packages sets up some sane solr settings that work for using it with Plone ootb. And to fix/improve a set up you need both: intermediate to good solr knowledge AND good Plone knowledge, which is less likely.

Upgrading to newer supported solr versions in the recipe is not usefull as long as collective.solr still mentions that it only works with Solr 4. Or is this not true anylonger and are people using collective.solr with Solr 6?

@tisto Can you or someone else who has the perission at least release b4 or release this to 6.0.0 so that the latest fixes are easily available for others?

[fredvd]

@saily @do3cc @mauritsvanrees @gforcada I had a chat with @tisto this morning about collective.recipe.solrinstance at the Plone Conf 2017 sprint in Barcelona.

One of the main pain points of supporting this recipe is that the recipe tries to map all solr settings into recipe variables, which is an extra layer and a burden to maintain. If we step away from this and provide a sane xml default file that works out of the box with Plone. When users are ready for customising and improving their solr setup they can change the xml and that's documented well by the solr project itself.

At that point they can also use collective.recipe.template or other buildout template solutions for variable substitution. Timo has very good knowledge of solr setup and can provide best practice solr configs for the major Solr versions that provide an out of the box starting point for Plone.

The recipe would then only need a variable for where to download solr from. If you want to customise the setup, we document how to use wget to get the default template out of the github repo and you can start customise that.
The only tricky part I see with this is that as a newbie you might want to control the listen port(s) of solr without touching the config file, but maybe that's settable from the command line startup.

Second issue is that your solr configuration with the indexes etc are somewhat tied to the collective.solr and also Plone version used. Therefore it might make more sense to move a more minimal solr setup with default config into the collective.solr package and make it available as an entry point. That way you could do something like:

[solr-build]
recipe = collective.solr:build
src = http://apache.rediris.es/lucene/solr/7.1.0/solr-7.1.0.tgz
port = 8983
# optional:
#config = path/to/you/custom/solr.xml

... which will install a minimal solr setup in parts, copy our default xml there and generate a start script in the bin directory. See plone.recipe.varnish which does something similar with entry points for build, config and the start script. If people are still in favour of maintaining or generating a default config with a lot of buildout recipe variables it would be possible to make a separate :configuration section for that.

By tying the install recipe into collective.solr the default configs can also be tailored to the solr version and settings collective.solr is using.

Any idea's / suggestions on this?

[saily]

@fredvd +1 to moving away from mapping all variables!

It's just to complex and requires a lot of effort. IMHO we should keep this recipe as simple as possible to maintain, because Solr people are moving pretty fast, and we can't keep track on updating stuff.

We should also consider a deprecation and versioning policy which makes it transparent to users which version of Solr is being used. My proposal is to map it directly to Solr versioning policy, so if you want to stick on Solr 6, use recipe >=6,<7.

A lot of complexity within this recipe comes from supporting different versions of Solr.

[mauritsvanrees]

Minimalising the recipe seems good yes.

Merging collective.solr and the recipe may be tricky. You can easily end up in a situation where you want to use version X of collective.solr because that one is compatible with your Plone version, and you want to use version Y (either higher or lower than X) because that one has the correct template for your wanted Solr version.

Having a collective.recipe.solr version Z that only works with Solr version Z seems like a good way to reduce complexity. Probably we should match major and minor versions. So recipe 7.0.x works with Solr 7.0.y, where x and y do not need to be the same: bug fix releases of Solr probably do not require changes to the recipe. If we are lucky, recipe 7.0.x also works with Solr 7.1.x, but we should make a 7.1 release anyway then.

[tisto]

@mauritsvanrees +1. What about creating a new c.r.solrinstance major version from the scratch then? If we do not have anybody who wants to support the old way of doing things that might be the way to go.

[mauritsvanrees]

From scratch seems good yes.

[tisto]

I went ahead and created a PoC for a new buildout recipe from the scratch:

https://github.com/kitconcept/kitconcept.recipe.solr

So far it only downloads Solr and creates a core. I will try to add a basic Solr configuration that works with different versions of Plone and Solr. My main use case is that I want to test collective.solr with the latest Solr releases, therefore I will focus on that first. Help is always welcome!