[REVIEW]: HyperNetX: A Python package for modeling complex network data as hypergraphs
Closed this issue · 121 comments
Submitting author: @brendapraggastis (Brenda Praggastis)
Repository: https://github.com/pnnl/HyperNetX
Branch with paper.md (empty if default branch): paper
Version: v2.2.0p
Editor: @danielskatz
Reviewers: @szhorvat, @IvanIsCoding, @drj11
Archive: 10.5281/zenodo.10795225
Status
Status badge code:
HTML: <a href="https://joss.theoj.org/papers/97a1ebb3cfc47070c672cb5e6f2c474f"><img src="https://joss.theoj.org/papers/97a1ebb3cfc47070c672cb5e6f2c474f/status.svg"></a>
Markdown: [](https://joss.theoj.org/papers/97a1ebb3cfc47070c672cb5e6f2c474f)
Reviewers and authors:
Please avoid lengthy details of difficulties in the review thread. Instead, please create a new issue in the target repository and link to those issues (especially acceptance-blockers) by leaving comments in the review thread below. (For completists: if the target issue tracker is also on GitHub, linking the review thread in the issue or vice versa will create corresponding breadcrumb trails in the link target.)
Reviewer instructions & questions
@szhorvat & @IvanIsCoding, your review will be checklist based. Each of you will have a separate checklist that you should update when carrying out your review.
First of all you need to run this command in a separate comment to create the checklist:
@editorialbot generate my checklist
The reviewer guidelines are available here: https://joss.readthedocs.io/en/latest/reviewer_guidelines.html. Any questions/concerns please let @danielskatz know.
✨ Please start on your review when you are able, and be sure to complete your review in the next six weeks, at the very latest ✨
Checklists
Hello humans, I'm @editorialbot, a robot that can help you with some common editorial tasks.
For a list of things I can do to help you, just type:
@editorialbot commands
For example, to regenerate the paper pdf after making changes in the paper's md or bib files, type:
@editorialbot generate pdf
Software report:
github.com/AlDanial/cloc v 1.88 T=0.53 s (189.1 files/s, 428840.5 lines/s)
-------------------------------------------------------------------------------
Language files blank comment code
-------------------------------------------------------------------------------
Python 48 2280 5245 5609
Jupyter Notebook 12 0 211574 1108
reStructuredText 21 389 358 818
Markdown 6 204 0 554
DOS Batch 1 34 2 227
TeX 1 23 0 206
YAML 6 37 29 178
make 2 34 22 67
INI 2 4 2 35
TOML 2 0 0 27
-------------------------------------------------------------------------------
SUM: 101 3005 217232 8829
-------------------------------------------------------------------------------
gitinspector failed to run statistical information for the repository
Wordcount for paper.md is 1206
Reference check summary (note 'MISSING' DOIs are suggestions that need verification):
OK DOIs
- 10.1007/978-3-030-48478-1_1 is OK
- 10.3390/math10050759 is OK
- 10.1109/hipc53243.2021.00045 is OK
- 10.1109/tvcg.2022.3153895 is OK
- 10.1140/epjds/s13688-020-00231-0 is OK
- 10.1371/journal.pcbi.1011190 is OK
- 10.1016/s0924-6509(09)x7013-3 is OK
- 10.1016/s0924-6509(08)x7007-2 is OK
- 10.1186/s12859-021-04197-2 is OK
- 10.48550/arXiv.2208.03103 is OK
- 10.1093/comnet/cnad019 is OK
- 10.48550/arXiv.2208.06894 is OK
- 10.1007/978-3-030-67318-5_25 is OK
- 10.1137/20M1355896 is OK
- 10.1007/978-3-030-71704-9_1 is OK
- 10.21105/joss.05162 is OK
- 10.1007/978-3-030-25070-6_9 is OK
- 10.1101/2023.09.29.560191 is OK
MISSING DOIs
- None
INVALID DOIs
- None
@szhorvat, @IvanIsCoding, and @drj11 - Thanks for agreeing to review this submission.
This is the review thread for the paper. All of our communications will happen here from now on.
As you can see above, you each should use the command @editorialbot generate my checklist to create your review checklist. @editorialbot commands need to be the first thing in a new comment.
As you go over the submission, please check any items that you feel have been satisfied. There are also links to the JOSS reviewer guidelines.
The JOSS review is different from most other journals. Our goal is to work with the authors to help them meet our criteria instead of merely passing judgment on the submission. As such, reviewers are encouraged to submit issues and pull requests on the software repository. When doing so, please mention openjournals/joss-reviews#6016 so that a link is created to this thread (and I can keep an eye on what is happening). Please also feel free to comment and ask questions on this thread. In my experience, it is better to post comments/questions/suggestions as you come across them instead of waiting until you've reviewed the entire package.
We aim for reviews to be completed within about 4 weeks. Please let me know if either of you require some more time. We can also use editorialbot (our bot) to set automatic reminders if you know you'll be away for a known period of time.
Please feel free to ping me (@danielskatz) if you have any questions/concerns.
@editorialbot add @drj11 as reviewer
@drj11 added to the reviewers list!
Review checklist for @drj11
Conflict of interest
- I confirm that I have read the JOSS conflict of interest (COI) policy and that: I have no COIs with reviewing this work or that any perceived COIs have been waived by JOSS for the purpose of this review.
Code of Conduct
- I confirm that I read and will adhere to the JOSS code of conduct.
General checks
- Repository: Is the source code for this software available at the https://github.com/pnnl/HyperNetX?
- License: Does the repository contain a plain-text LICENSE file with the contents of an OSI approved software license?
- Contribution and authorship: Has the submitting author (@brendapraggastis) made major contributions to the software? Does the full list of paper authors seem appropriate and complete?
- Substantial scholarly effort: Does this submission meet the scope eligibility described in the JOSS guidelines
- Data sharing: If the paper contains original data, data are accessible to the reviewers. If the paper contains no original data, please check this item.
- Reproducibility: If the paper contains original results, results are entirely reproducible by reviewers. If the paper contains no original results, please check this item.
- Human and animal research: If the paper contains original data research on humans subjects or animals, does it comply with JOSS's human participants research policy and/or animal research policy? If the paper contains no such data, please check this item.
Functionality
- Installation: Does installation proceed as outlined in the documentation?
- Functionality: Have the functional claims of the software been confirmed?
- Performance: If there are any performance claims of the software, have they been confirmed? (If there are no claims, please check off this item.)
Documentation
- A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
- Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.
- Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).
- Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?
- Automated tests: Are there automated tests or manual steps described so that the functionality of the software can be verified?
- Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support
Software paper
- Summary: Has a clear description of the high-level functionality and purpose of the software for a diverse, non-specialist audience been provided?
- A statement of need: Does the paper have a section titled 'Statement of need' that clearly states what problems the software is designed to solve, who the target audience is, and its relation to other work?
- State of the field: Do the authors describe how this software compares to other commonly-used packages?
- Quality of writing: Is the paper well written (i.e., it does not require editing for structure, language, or writing quality)?
- References: Is the list of references complete, and is everything cited appropriately that should be cited (e.g., papers, datasets, software)? Do references in the text use the proper citation syntax?
Review checklist for @szhorvat
Conflict of interest
- I confirm that I have read the JOSS conflict of interest (COI) policy and that: I have no COIs with reviewing this work or that any perceived COIs have been waived by JOSS for the purpose of this review.
Code of Conduct
- I confirm that I read and will adhere to the JOSS code of conduct.
General checks
- Repository: Is the source code for this software available at the https://github.com/pnnl/HyperNetX?
- License: Does the repository contain a plain-text LICENSE file with the contents of an OSI approved software license?
- Contribution and authorship: Has the submitting author (@brendapraggastis) made major contributions to the software? Does the full list of paper authors seem appropriate and complete? See #6016 (comment)
- Substantial scholarly effort: Does this submission meet the scope eligibility described in the JOSS guidelines
- Data sharing: If the paper contains original data, data are accessible to the reviewers. If the paper contains no original data, please check this item.
- Reproducibility: If the paper contains original results, results are entirely reproducible by reviewers. If the paper contains no original results, please check this item.
- Human and animal research: If the paper contains original data research on humans subjects or animals, does it comply with JOSS's human participants research policy and/or animal research policy? If the paper contains no such data, please check this item.
Functionality
- Installation: Does installation proceed as outlined in the documentation?
- Functionality: Have the functional claims of the software been confirmed?
- Performance: If there are any performance claims of the software, have they been confirmed? (If there are no claims, please check off this item.)
Documentation
- A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
- Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.
- Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).
- Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?
- Automated tests: Are there automated tests or manual steps described so that the functionality of the software can be verified?
- Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support See pnnl/HyperNetX#144
Software paper
- Summary: Has a clear description of the high-level functionality and purpose of the software for a diverse, non-specialist audience been provided?
- A statement of need: Does the paper have a section titled 'Statement of need' that clearly states what problems the software is designed to solve, who the target audience is, and its relation to other work?
- State of the field: Do the authors describe how this software compares to other commonly-used packages? See #6016 (comment)
- Quality of writing: Is the paper well written (i.e., it does not require editing for structure, language, or writing quality)?
- References: Is the list of references complete, and is everything cited appropriately that should be cited (e.g., papers, datasets, software)? Do references in the text use the proper citation syntax?
Review checklist for @IvanIsCoding
Conflict of interest
- I confirm that I have read the JOSS conflict of interest (COI) policy and that: I have no COIs with reviewing this work or that any perceived COIs have been waived by JOSS for the purpose of this review.
Code of Conduct
- I confirm that I read and will adhere to the JOSS code of conduct.
General checks
- Repository: Is the source code for this software available at the https://github.com/pnnl/HyperNetX?
- License: Does the repository contain a plain-text LICENSE file with the contents of an OSI approved software license?
- Contribution and authorship: Has the submitting author (@brendapraggastis) made major contributions to the software? Does the full list of paper authors seem appropriate and complete?
- Substantial scholarly effort: Does this submission meet the scope eligibility described in the JOSS guidelines
- Data sharing: If the paper contains original data, data are accessible to the reviewers. If the paper contains no original data, please check this item.
- Reproducibility: If the paper contains original results, results are entirely reproducible by reviewers. If the paper contains no original results, please check this item.
- Human and animal research: If the paper contains original data research on humans subjects or animals, does it comply with JOSS's human participants research policy and/or animal research policy? If the paper contains no such data, please check this item.
Functionality
- Installation: Does installation proceed as outlined in the documentation?
- Functionality: Have the functional claims of the software been confirmed?
- Performance: If there are any performance claims of the software, have they been confirmed? (If there are no claims, please check off this item.)
Documentation
- A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
- Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.
- Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).
- Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?
- Automated tests: Are there automated tests or manual steps described so that the functionality of the software can be verified?
- Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support
Software paper
- Summary: Has a clear description of the high-level functionality and purpose of the software for a diverse, non-specialist audience been provided?
- A statement of need: Does the paper have a section titled 'Statement of need' that clearly states what problems the software is designed to solve, who the target audience is, and its relation to other work?
- State of the field: Do the authors describe how this software compares to other commonly-used packages?
- Quality of writing: Is the paper well written (i.e., it does not require editing for structure, language, or writing quality)?
- References: Is the list of references complete, and is everything cited appropriately that should be cited (e.g., papers, datasets, software)? Do references in the text use the proper citation syntax?
I have reviewed the software package and preliminarily I believe it would be a great addition to the papers published at the Journal.
Software
With regards to the software, I do not have objections. I believe HyperNetX is a significant scholarly contribution and that it is as mature as if not more mature than other packages published in this journal.
From a Software Engineering point of view, I also found the that the test suite of the package meets the standards for publication. I was able to install the software and run the tests without issues. I was also able to generate a code coverage report. There are some warnings when running the tests and the coverage for the Drawing module is on the lower side. But overall the software is tested and Continuous Integration is in place to detect regressions, which is nice.
Documentation
For the documentation I focused more on the provided Jupyter Notebooks and A Gentle Introduction to Hypergraph Mathematics provided in the docs. There are also documentations for every function and class, but I admit I haven't scrutinized each method and looked at the greater picture.
As a reviewer with a greater emphasis on graphs than hypegraphs, I found that the "gentle" introduction answered most of the questions for the statement of need and defined the terminology for the package.
The Jupyter Notebooks available in Colab were also a pleasant surprise. I think they exemplify how to use the package and highlight the visualization features that come with HyperNetX.
Paper
This is the only section where I think there is feedback to give to the authors from my side. Namely:
- The "HNX-Widget: Interactive Diagrams of Hypergraphs" is my least favourite section of the paper. I do not believe it aggregates much given the word limit recommended by JOSS. I suggest using an image with a detailed caption to convey the message better
- The authors should cite
pandasfollowing the guidelines of the package - The authors should cite
NetworkXfollowing the guidelines of the package
👋 @brendapraggastis - while two reviews are currently in progress, you could respond to (or act on) @IvanIsCoding's comments and issue during this time.
Thanks @IvanIsCoding and @danielskatz . We are addressing your comments and should be able to push paper changes in the next week or so.
Sorry about the long absence. If all goes according to plan I hope to finish the review by Dec 14.
@IvanIsCoding Thanks for the detailed review. I've incorporated your suggesttions in the paper in https://github.com/pnnl/HyperNetX/tree/paper/HNX_Paper.
@editorialbot generate pdf
@brendapraggastis - you could issue this command too, if you make further changes. I'll also check the references again next. Commands need to be the first thing in a comment
@editorialbot check references
Reference check summary (note 'MISSING' DOIs are suggestions that need verification):
OK DOIs
- 10.1007/978-3-030-48478-1_1 is OK
- 10.3390/math10050759 is OK
- 10.1109/hipc53243.2021.00045 is OK
- 10.1109/tvcg.2022.3153895 is OK
- 10.1140/epjds/s13688-020-00231-0 is OK
- 10.1371/journal.pcbi.1011190 is OK
- 10.1016/s0924-6509(09)x7013-3 is OK
- 10.1016/s0924-6509(08)x7007-2 is OK
- 10.1186/s12859-021-04197-2 is OK
- 10.48550/arXiv.2208.03103 is OK
- 10.1093/comnet/cnad019 is OK
- 10.48550/arXiv.2208.06894 is OK
- 10.1007/978-3-030-67318-5_25 is OK
- 10.1137/20M1355896 is OK
- 10.1007/978-3-030-71704-9_1 is OK
- 10.21105/joss.05162 is OK
- 10.1007/978-3-030-25070-6_9 is OK
- 10.1101/2023.09.29.560191 is OK
- 10.5281/zenodo.3509134 is OK
- 10.25080/Majora-92bf1922-00a is OK
MISSING DOIs
- None
INVALID DOIs
- None
@brendapraggastis - if an author has an ORCID, please enter it. If not, please remove the line about it.
@editorialbot generate pdf
@danielskatz I put the ORCID numbers back in. I believe it is up-to-date now.
On it! Thanks for the nudge.
I have completed my review. While i have not poked into every corner of functionality described in the paper, i have poked around enough to be satisfied that the functionality of HyperNetX is substantially as described in its documentation. I find no further reason to delay publication, and recommend that this paper be published by JOSS.
I find the subject to be quite novel and interesting (particularly coming from a more trad graph theory background), and i liked the inclusion of Les Misérables in the tutorials. The package as a whole is made to a high standard, and integrates well with the established Python ecosystems. It is a credit to both the Python and the scientific programming community. I commend the authors for their efforts.
My only real substantial comment, and it is relatively minor, is that i would spent some time on your very long pylint and my own particular bugbear: bare-except.
thanks @drj11!
👋 @szhorvat - how is your review coming?
Thanks for the reminder @danielskatz. I was travelling in China for 3 weeks, and accessing GitHub, or even PyPI, turned out to be much less convenient than I expected due to the GFW. I got home on Monday and back to work now.
👋 @szhorvat - I'm just checking in post-holidays on how your review is going. I look forward to seeing progress, or some discussion about what's blocking you. (Given that we have two complete reviews, we could move forward, but since you started on this, I want to give you the chance to finish and add your contributions to the process, assuming this is still what you want to do.)
@IvanIsCoding I have addressed the issues that your raised and merged in the changes in both the paper and master branch; the changes can be referenced via the commit message. Any commit message prefixed with HYP-364 refers to the requested changes for this review. @szhorvat The paper branch has been updated and ready for review. Please let me know if you need anything else or have any questions.
@editorialbot generate pdf
Note to @bonicim - you could also run this command if there are further changes
👋 @IvanIsCoding & @drj11 - it looks like you've now checked off all the items on your reviews. Can you confirm that you are ok with this being published now?
👋 @szhorvat - How is your review coming? When do you think it might be complete?
@danielskatz my review is (still) complete, and i am okay with proceeding to publish.
I am working on it at the moment and have the full day tomorrow set aside for this. So far it looks like a very solid package.
@brendapraggastis When citing igraph in the paper, could you please spell it with a lowercase "i" and also include https://arxiv.org/abs/2311.10260 (which will eventually become the main citation point, replacing the old DOI-less paper)?
@szhorvat Thanks for the careful read. We've assigned most of your issues to be addressed either in a hotfix to the documentation (as in the error messages for empty hypergraphs, Issue #137 ) or a future release. Are there any specific issues you see that you believe must be remedied before publishing in JOSS?
@brendapraggastis Let's set the end of the week (Sunday) as the time point for finalizing the review. I'm sorry that this took so long. I am still playing with the library and will open issues until then.
Could you please answer here? pnnl/HyperNetX#135 This is not something for you to fix. I just need to know the best channels to ask for help.
You should treat all issues I open as suggestions for now.
Something that I do believe should be fixed in the paper (not library) is a better summary of related and alternative software. At the moment you are simply referencing a different paper that is: (1) over two years old in a rapidly developing area (2) is not actually a review paper on the topic (3) is paywalled and I cannot access it (can you email it to me please?) I would expect not just a list of alternatives, but some sort of useful guidance for users on what packages to look at given their application area, and how HyperNetX differs from these. I understand that JOSS uses a very constraining word limit, but you are not at the word limit, and if the paper is to be useful to readers, this is an important area to improve. Looking at how other related software handle this, XGI has a list of related/alternative packages right in their README: https://github.com/xgi-org/xgi
@szhorvat I have answered pnnl/HyperNetX#135. You can now start a general discussion here.
Before I can check off this item:
- Contribution and authorship: Has the submitting author (@brendapraggastis) made major contributions to the software? Does the full list of paper authors seem appropriate and complete?
I wanted to note that only 3 out of the 8 authors seem to have made commits to the repository, https://github.com/pnnl/HyperNetX/graphs/contributors (unless I failed to match up some GitHub user names). Authorship policy is up to the journal (and personally I certainly do not think that authorship should require code contributions). Therefore I leave it up to the editors whether they need further clarifications on this.
Before I can check off this item:
- Contribution and authorship: Has the submitting author (@brendapraggastis) made major contributions to the software? Does the full list of paper authors seem appropriate and complete?
I wanted to note that only 3 out of the 8 authors seem to have made commits to the repository, https://github.com/pnnl/HyperNetX/graphs/contributors (unless I failed to match up some GitHub user names). Authorship policy is up to the journal (and personally I certainly do not think that authorship should require code contributions). Therefore I leave it up to the editors whether they need further clarifications on this.
@szhorvat All listed authors have contributed to the repository. Ongoing code adjustments and documentation must be reviewed by our company before public release so the true commit history isn't visible on Github.
It looks like there is some discussion/work going on between @brendapraggastis @bonicim and @szhorvat, so I think we're waiting for these issues to get resolved before we can proceed. If anyone disagrees, please say so.
@danielskatz and @szhorvat - it looks like the only thing pending is a better description of other hypergraph libraries. This will take some time depending on how in depth a comparison we make.
@szhorvat suggests "I would expect not just a list of alternatives, but some sort of useful guidance for users on what packages to look at given their application area, and how HyperNetX differs from these."
I want to make sure we fairly represent the other libraries and a deep dive into their inner workings will take time. I will discuss this with my team and follow up with you later this week.
@brendapraggastis - thanks for your update. However, in addition to what you say, I also see a 11 open issues that are tagged with this review, I think all from @szhorvat. It's unclear to me how many of these are blockers for @szhorvat (vs suggestions for the future), but given the unchecked review items in @szhorvat's checklist, I assume some of them do need to be addressed. @szhorvat - perhaps you can make it clear which of these issues fall into which category?
I would like to suggest fixing the following before publication:
- pnnl/HyperNetX#145 (quick and easy doc fix)
- pnnl/HyperNetX#144 I consider the contributor guide important for any open source project.
- pnnl/HyperNetX#143 : This can't be fixed quickly, and it wouldn't be reasonable to ask for this to be fixed before publishing. But given that this feature is prominently mentioned in the paper, and currently it is quite difficult to get it working, could the authors please comment on what the plans are going forward? Will it be supported and made easy to use, or will it be kept in its current state or dropped?
- pnnl/HyperNetX#142 This is not a major issue, but again it is a quick fix, and I'd like to see it corrected for the sake of user friendliness.
I want to make sure we fairly represent the other libraries and a deep dive into their inner workings will take time.
To be clear, I am not asking for a deep dive, after all this is not a review paper. However, currently there is no guidance at all for readers about what alternatives exist and what makes HNX different. There is a figure (Fig 2) with no main text reference, and a citation to Kurte et al, which I still wasn't able to access.
With that out of the way, I wanted to say that HNX looks like a great package that is breaking new ground. Modelling systems using higher order networks is still a relatively new, and quickly developing approach. I enjoyed exploring HNX and I'm looking forward to seeing significant development in the coming years.
@szhorvat https://rdcu.be/dw897 I was able to share the Kurte et al. paper. I hope that helps
- [joss] Testing instructions are incomplete in the documentation pnnl/HyperNetX#145 (quick and easy doc fix)
Will fix.
- [joss] Contributor guide should be more specific, and probably shouldn't suggest Rust as the language pnnl/HyperNetX#144 I consider the contributor guide important for any open source project.
Will fix.
- [joss] HyperNetX Widget not working pnnl/HyperNetX#143 : This can't be fixed quickly, and it wouldn't be reasonable to ask for this to be fixed before publishing. But given that this feature is prominently mentioned in the paper, and currently it is quite difficult to get it working, could the authors please comment on what the plans are going forward? Will it be supported and made easy to use, or will it be kept in its current state or dropped?
The HyperNetX widget is currently in beta with limitations on the Jupyter environment in which it may be used. It is being actively worked on. Look for improvements and an expanded list of usable environments in a future release.
In addition, updates to the documentation are currently in progress and are planned to be published in the next week or so.
- [joss] More clarity on (optional) dependencies pnnl/HyperNetX#142 This is not a major issue, but again it is a quick fix, and I'd like to see it corrected for the sake of user friendliness.
Will fix.
@brendapraggastis & @bonicim - How are your updates coming?
danielskatz
Updates are almost done; we expect to have them pushed to master fairly soon. Will post an update ASAP.
@danielskatz Updates pushed to master branch and reflected in latest release, 2.1.4. PyPi has also been updated.
Additional comments on other hypergraph libraries are forthcoming.
@bonicim - When you are done with the changes, please use the command @editorialbot generate pdf to make a new PDF. editorialbot commands need to be the first entry in a new comment.
@editorialbot generate pdf
I will look into the pdf generation error. I have an idea of what happened; will rerun the command once I've updated the paper branch.
@editorialbot generate pdf
@editorialbot generate pdf
Added section on comparison with other hypergraph libraries.
Let me know when you feel it's ready.
EDIT: Can you please make sure the references are added before the fullstop within the section about other hypergraph libraries?
@szhorvat Yes, I am looking at our latest copy. I think it is just a problem with the markdown used. Will let you know when it is fixed.
@editorialbot generate pdf
@szhorvat and @danielskatz the latest version has references to the three most contemporary libraries to HNX that we know of along with their references. Hopefully this version is acceptable for publication.
@szhorvat - when you get a chance, please let us know if you agree with ☝️
I was still waiting for a response on a minor comment here: pnnl/HyperNetX#143 (comment)
Perhaps other reviewers can comment on whether what is suggested (i.e. install everything in the conda base environment) is considered good practice. Python technical details are not my area of expertise.
My guiding principle with these comments was to try to make sure that academic users, many of whom might not be very technical, will be well-served by the documentation.
@IvanIsCoding, @drj11 - any thoughts on this? See the issue in the comment above for the discussion
@editorialbot generate pdf
@brendapraggastis When citing igraph in the paper, could you please spell it with a lowercase "i" and also include https://arxiv.org/abs/2311.10260 (which will eventually become the main citation point, replacing the old DOI-less paper)?
Could you please correct the uppercase "i" to lowercase before finalizing? I'll write up my report soon.
This is my final report. Please note that I'll be away, and unresponsive, during the following week.
HNX is an ambitious package that is breaking new ground within a rapidly developing field. I am happy to see such packages published in JOSS, and I am looking forward to seeing further HNX development.
Some suggestions to the authors:
Openness of development HNX is open-source in the sense that it has an open source license, and that it is technically accepting contributions on GitHub. However, the development process is opaque, and does not project an image that encourages contributions. Looking at the list of PRs, the majority of non-trivial ones were closed with a comment that they were handled internally. There are very few truly external contributions, and those that exist did not leave a mark on the git commit history (i.e. the name of the actual contributor wasn't recorded). I think HNX could truly benefit from transitioning to more welcoming practices. Consider making development more open and especially crediting contributors in some way.
HNX widget The HNX widget is featured prominently in the paper, but it turned out to function only under very specific circumstances. The authors improved the documentation and stated that the widget is still in beta and under active development. However, there hasn't been any public development (or release) in 3 years. Is this another symptom of the behind-closed-doors development practices? I am hoping to see significant improvements in the near future.
Documentation I found the documentation to be a bit terse for a scientific package (see e.g. pnnl/HyperNetX#141) I know that keeping documentation minimal is very common with Python packages—HNX is not different in this respect. But give the target audience (researchers) and the mathematically non-trivial topics the package deals with, I think that putting more effort into documentation would be one of the biggest improvements HNX could make at the moment. Initiatives like pyOpenSci are already pushing for better documentation practices, I suggest taking a look at their guidelines, or just looking at common practices used in the communities of more scientifically oriented languages (e.g. R or Mathematica).
@bonicim @brendapraggastis Thank you for the patience. I hand over to @danielskatz now.
@IvanIsCoding, @drj11 - any thoughts on this? See the issue in the comment above for the discussion
For the record on the Jupyter Widget debate.
With regards to packaging, ideally users would only pip install hnxwidget and just use it. But I myself have experienced issues with Jupyter Widgets as a user so I imagine that on the packaging side it can be challenging as well. A solution using both pip and conda is not ideal, so my suggestion is to package hnxwidget for Conda.
@brendapraggastis When citing igraph in the paper, could you please spell it with a lowercase "i" and also include https://arxiv.org/abs/2311.10260 (which will eventually become the main citation point, replacing the old DOI-less paper)?
Could you please correct the uppercase "i" to lowercase before finalizing? I'll write up my report soon.
I think this is one is on the journal, the template capitalizes everyword. The author's bibtex is what you would expect.. I think if they switch to use {i}graph in Bibtex it will preserve the capitalization?
I think if they switch to use
{i}graphin Bibtex it will preserve the capitalization?
Yes, that works, but I meant to refer to the mention in the main text :-)
Also, to make sure there is no confusion, I do recommend publishing.
@IvanIsCoding - having now had all the reviews finish, we can go ahead and publish. Let me know if you want to make and changes at this point based on @szhorvat's comments. Once you are ready, I'll proofread the paper, and ask you to perform some archiving steps, then I'll finish the process.
@IvanIsCoding - having now had all the reviews finish, we can go ahead and publish. Let me know if you want to make and changes at this point based on comments. Once you are ready, I'll proofread the paper, and ask you to perform some archiving steps, then I'll finish the process.
I have no objections for the software. For the paper, my only objections are minor igraph citation details. I agree with the other reviewer on the spelling, it is an easy change to make. Another factor that I agree is that igraph has recently posted https://arxiv.org/abs/2311.10260, so I do agree with @szhorvat's comment to cite the pre-print and incorporate the decade of work spent improving the package.
sorry @IvanIsCoding - I meant to address this comment to @brendapraggastis
Let me know if you want to make and changes at this point based on @szhorvat's comments. Once you are ready, I'll proofread the paper, and ask you to perform some archiving steps, then I'll finish the process.
@danielskatz @IvanIsCoding @szhorvat Thanks for catching the Igraph error. We will fix that and the references this week and notify you as soon as the final is ready.
@editorialbot generate pdf
@danielskatz @IvanIsCoding @szhorvat igraph typo is fixed and igraph reference added.
@danielskatz @IvanIsCoding @szhorvat igraph typo is fixed and igraph reference added.
I believe we are good to go for a final proof. The footnote references will be updated by JOSS, correct?
yes, and thanks. I'll do the proof-reading a little later today
I've now suggested a bunch of minor changes in pnnl/HyperNetX#150 - please merge this, or let me know what you disagree with, then we can continue the process
@danielskatz The typo fix went in the wrong place. a95c8c8 put it in a sequence of terms, which I don't think is what you meant. I went through and made sure the punctuation after the citations were correct and in the correct place. @bonicim will push the fix this morning.
Sorry about that and thanks for fixing it.
@editorialbot generate pdf
@brendapraggastis - can you please check to make sure this is ok now?
@brendapraggastis - can you please check to make sure this is ok now?
@danielskatz The punctuation changes are reflected in the latest document. I don't see anything else, so hopefully we are good to go now.
Thanks. The next steps are for you to:
- Make a tagged release of your software, and list the version tag of the archived version here.
- Archive the reviewed software in Zenodo or a similar service (e.g., figshare, an institutional repository)
- Check the archival deposit (e.g., in Zenodo) has the correct metadata. This includes the title (should match the paper title) and author list (make sure the list is correct and people who only made a small fix are not on it). You may also add the authors' ORCID.
- Please list the DOI of the archived version here.
I can then move forward with accepting the submission.
@danielskatz I have created a tagged release, v2.2.0p. The rest of the required steps are currently in progress.
@editorialbot set v2.2.0p as version
Done! version is now v2.2.0p
Software has been stored on Zenodo at https://zenodo.org/records/10790797
DOI: 10.5281/zenodo.10790797
@editorialbot set 10.5281/zenodo.10790797 as archive
Done! archive is now 10.5281/zenodo.10790797
@editorialbot recommend-accept