Tracking issue: JOSS paper

[EwoutH]

Overview

This tracks the development of the Mesa JOSS paper that aims to provide a comprehensive overview of Mesa in its current state (Mesa 3). The paper will live on the paper_2024 branch.

Authors/maintainers, feel free to edit this tracking issue directly and open PRs into the paper_2024 branch.

Main Tasks

Content Structure & Organization

• Rewrite paper to focus on Mesa as a complete framework rather than centering on 3.0 changes (@EwoutH) paper: Restructure to present Mesa as complete framework #2560
• Add introductory paragraph about agent-based modeling
• Restructure content into three main sections:
  • ABM Framework (agents/management, spaces, time management)
  • Visualization
  • Experimentation
• Create updated component diagram similar to previous Mesa papers
• Add Acknowledgements section

Technical Elements

• Integrate experimental features into main narrative sections
• Review and update discussion of spaces (focusing on new-style spaces)

References & Citations

• Add more application references (paper: Include some highly cited papers that used Mesa #2572)
• Include Mesa-Geo paper citation
• Review and incorporate suggested bibtex entries
• Add additional relevant references

Author Management

• Collect remaining ORCIDs and confirm affiliations Paper: Add affiliations #2567

Next Steps

• 1. Merge current PR into paper_2024 branch
• 2. Implement standalone paper rewrite
• 3. Individual PRs can be opened against paper_2024 branch for specific improvements

Coordination

• Authors can submit changes via PR or provide feedback through GitHub suggestions

Notes

• Paper should be readable standalone without requiring familiarity with previous Mesa papers
• Focus on presenting Mesa in its current form while mentioning 3.0 features in context
• Consider 5-year publication cycle (previous papers: 2015, 2020, likely 2025)

Review process

• [PRE REVIEW]: Mesa 3: Agent-based modelling with Python in 2024 openjournals/joss-reviews#7620

[EwoutH]

On Jackie's request, there is now a Google Docs in which we continue to edit, suggest and review: https://docs.google.com/document/d/1HdoNDecjRvvmYhMyr9lZqPj3mmTOa3lrFv1a8_NCk38/edit?usp=sharing

@projectmesa/maintainers would appreciate everyone's review!

[jackiekazil]

It looks like it is currently too long-- 1500+ and it needs to be <1000. I am going to work on suggesting cuts to trim down.

[quaquel]

It seemed API docs are not permitted so most example code snippets might have to be removed.

[EwoutH]

I checked some recent JOSS papers and some certainly include code:

• 
• 
• 

I also highly believe in showing by example. But maybe we could cut the code blocks down to the bare essentials.

@jackiekazil we need to trim down a bit, I don't know if they count code. If they don't, we are already in the right ballpark.

Also we can just submit at some point and see what the reviewers think of it. The review guideline states that preparing a manuscript can take as little as one hour:

If your software is already well documented then paper preparation should take no more than an hour.

While probably optimistic / best case, we don't need to overdo this.

[jackiekazil]

I am struggling with the current paper structure without submitting something that is repetitive with previous papers, but also provides enough background for a reader to understand advancements. I reached out to JOSS to try to get examples of papers that are version updates to see how others structured theirs -- openjournals/joss#1403 -- specifically what level of context detail did they include?

I am interested to see what they highlight, because I can only see two paths right now -- 1) A lot of background info, which is repetitive with previous publications and will run long, but offers context for the advancements OR 2) A paper is almost a diff but in natural language -- where context maybe lacking.

[quaquel]

I am struggling with the current paper structure without submitting something that is repetitive with previous papers,

I am not sure I understand the concern. I see this paper as the suggested citation for users of MESA and their academic entry point. Thus I want this paper to be independently readable. I doubt many readers will go back to earlier papers anyway, so why worry about it?

[EwoutH]

@jackiekazil @quaquel and @wang-boyu thanks for some nice comments and suggestions! I resolved, accepted or commented on most of them.

Personally, while I still am fine with both options, I think a current snapshot paper is more useful than an update paper. It provides the context and scaffolding, and now reads like a nice 101. Most modellers/programmers already can find more than sufficient resources on Mesa, this paper is for academics without GitHub accounts who prefer reading papers over tutorials or docs.

Length wise, I think we're good. It's close enough when we present the code blocks as "figures".

I would also appreciate it if everybody could confirm their name, ORCID and affiliations in #2567.

[jackiekazil]

I was struggling, but reflecting, it was more with the JOSS length limitation and the complexity of the current state with updates of Mesa. Citation -- sure... but the paper aims to highlight new work. Otherwise, there is no paper. But to highlight the new work, you need to explain the foundation of the baseline (previous work) on which it stands because advances and changes don't make sense out of context without the foundation to explain them -- which for JOSS and the requisite brevity, I think that means we are aligned in the direction.

We will see if JOSS responds, but we can submit without a response and maybe just reference it if length comes up. I want to take at least one more pass-through (planning on Sunday night - depending on personal life demands).

[EwoutH]

• Current PDF of the paper.

@jackiekazil please review this version, the docs is not fully up to date anymore.

[jackiekazil]

oh no... I just saw this after adding another review to doc. 😭

[EwoutH]

@jackiekazil No worries, I ported your suggestions from the docs to our paper.

---

Submitted to JOSS!

If we want to make any more changes, we can just keep updating our paper_2024 branch.

[EwoutH]

They're moving quite fast at JOSS, our paper is already in pre-review:

• [PRE REVIEW]: Mesa 3: Agent-based modelling with Python in 2024 openjournals/joss-reviews#7620

I'm currently resolving the points raised by the editor, feel free to subscribe on that issue and follow the process.

[jackiekazil]

Thank you for the integration, but I didn't think we were ready to submit yet there were pending questions -- especially with the experimental stuff. I do not feel comfortable with the current submission.

[EwoutH]

Sorry, I didn’t get that sentiment from your review.

The two experimental parts that are note in the paper:

• Cell Space. API is fully fledged out, included in multiple example models, and ready to be stabilized. Old spaces are already maintenance only. We’re dotting the I’s at the point. See Tracking issue: Stabilizing the Cell Space #2519.
• DEVS: mentioned less extensively in the paper, and also has been included for more than a year.

Also let’s consider what this paper is: it’s a brief intro into Mesa and its capabilities. Thus, we want to show our wide range of capabilities. It’s not a tutorial, reference documentation or anything else.

Both features are more than stable enough to be used, and we’re convinced they add value so we’re not removing them. We could mention a bit more extensively that both are in the process of being stabilized.

Please let me know how you want to approach this. I could go through your docs comments again and try to integrate those, but if you would like to do it differently let me know.

CC @quaquel.

[quaquel]

I, too, am a bit surprised that it was already submitted, but that was more because of @Corvince not having a chance yet to look at it. Content-wise, however, there were no outstanding objections from my side. As indicated by @EwoutH, the experimental stuff is either about to become stable in 3.2, or mentioned only in passing (while having effectively a stable api).

[jackiekazil]

I will catch up on notes later -- heard back from rht via email - confirmed he would like to be an author. Forwarded to @EwoutH for awareness/handling since I am not able to right at this moment.

[EwoutH]

@jackiekazil thanks for sending a second mail, and @rht thanks for getting back. I added you as an author in 6a743f6. I will check with JOSS if they're okay with an pseudonym and if they are willing to waive the country requirement in the affiliation.

I, too, am a bit surprised that it was already submitted, but that was more because of @Corvince not having a chance yet to look at it.

Yeah I should have at least communicated that more clearly. JOSS is a bit different than most journals, their review process is iterative by design.

JOSS reviews are iterative and conversational in nature. Reviewers are encouraged to post comments/questions/suggestions in the review thread as they arise, and authors are expected to respond in a timely fashion.

So of course the core of the paper has to be ready for review, but details can be tuned in the process. Now at least the process of first finding an editor, and then finding reviewers can start.

[jackiekazil]

@EwoutH Part of my frustration is compounding factors around how poorly this was executed and hastily done. I am trying to refrain from rehashing/laundry listing here, but we need to do a retro on this and discuss how we can improve the process of something like this in the future.

To the current pending points:

RE: Experimental features - the paper 3.11+ -- but those features are experimental -- so they should be labeled as such. What you suggested regarding this is fine. (Would you be able to make this edit?)

RE: Edits to the paper—I still want to do an additional review or two for polish. I will submit it against paper_2024. After they are merged, are they automatically integrated into the submission?

[EwoutH]

Sorry to hear that. I’m willing to participate in a retro.

Experimental features

I will prepare an edit.

Edits to the paper

Yes, any PR/commits to paper_2024 get automatically integrated in the submission.

[EwoutH]

@jackiekazil I regularly send messages on Matrix what my approach/timeline would be and if anyone objects. Yesterday I also send you a personal message.

I think part of the friction comes from that communication not reaching you, giving you no chance to consider or react to it, and me no chance to receive that information and reconsider or adjust. Would you like to communicate in another way or on another platform?

We’re in especially difficult time zones, you just went to bed before I now just woke up. We can’t do anything about that, but it’s one of the constraints we have to work with.

[jackiekazil]

@EwoutH - I think this involves a further discussion (aka retro) -- which we can do on any platform -- I would prefer to do it in via a meeting only because I think tone of voice is vital in cases like this.

RE: Async comms --There are a couple of layers to problem. This added to it - and agree -- it would be helpful to discuss patterns around it.

[jackiekazil]

@EwoutH
RE: #2587
I hastily pushed this through to avoid holding it up and didn't consider the context around it.

The new cell-based spaces are feature complete but still in experimental state. A new ContinuousSpace implementation is in active development.

The cell-based spaces and continuous spaces existed before. But the newer versions of them are in an experimental state -- can you add clarity on what the advancements are? (Or if you want to dump something here - I can add it. I am about to touch next.)

[EwoutH]

can you add clarity on what the advancements are?

Given that we would have to go quite in depth, I think that would conflict of the nature of JOSS. Their paper format gives a brief introduction on a package, not a in-depth dive into individual features. Their philosophy is that that should be in your repository and documentation, since that is continuously updated and thus doesn’t get outdated like a paper.

A JOSS paper is a hook for “explore our package, repo and docs”. The goal is to lure people in (and have something citeable).

[quaquel]

The cell-based spaces and continuous spaces existed before. But the newer versions of them are in an experimental state -- can you add clarity on what the advancements are? (Or if you want to dump something here - I can add it. I am about to touch next.)

My personal take is the following. In 3.2, we plan to make cell spaces stable. Why worry at all about the fact that in 3.1.1. they are still in the experimental namespace? We already stated with the release of MESA 3 that old-style cell spaces are maintenance only. In 3.2, there will be a PendingDeprecationWarning on them. Likewise, in 3.2, all basic examples will shift to cell spaces. Since it is plausible that 3.2 will come out within the next two months, why write the paper such that it is outdated by the time it is accepted?

[jackiekazil]

The clarification that I am looking for it -- cell based spaces already exist in Python, but in the paper we are saying they are experimental. This is confusing, because they are already exist. That is the type of clarification that I am looking for -- 1 to 2 lines on how the cell based spaces changed for the better, which is why they are in experimental.

[quaquel]

I am sorry but I don't follow. Where do we say in the paper that cell-based spaces already exist in Python? We explain how discrete/cell-based spaces are implemented in MESA, but don't make wider claims regarding Python. As indicated, I would prefer to completely drop lines 74-75 from the latest pdf.

[jackiekazil]

@quaquel I think you were right about 74-75 not advancing the paper well. I added minimal clarifying language, which I think is important, especially with the imports in the latest PR - #2597

[EwoutH]

JOSS has found an editor to pick up our paper! openjournals/joss-reviews#7620 (comment)

Now the search for reviewers has started.