visortelle/haskell-spotlight

⏳ Changelog

visortelle opened this issue · 64 comments

⏳ Changelog

Day 1.

Home page layout inspired by the Rust's crates.io

day-1

Day 2.

"Most Downloaded" and "Recently Updated" package lists at home page.

Screen.Recording.2022-01-01.at.5.58.59.PM.mov

Day 2. A simple search in Hackage.

Screen.Recording.2022-01-01.at.10.32.05.PM.mov

Day 3. Implemented search in Hoogle.

Screen.Recording.2022-01-02.at.3.14.51.PM.mov

Day 3. More informative layout for Hoogle search results.

Screen.Recording.2022-01-02.at.6.37.08.PM.mov

Great so see some work on this issue, thank you! One thought I had staring at the screenshots: the big bold numbers on the right of the main part of the page (Downloads, Packages Published) look pretty useless to me, and I'd hope to have little noice on the main page. I understand it's modeled after crates.io, and I appreciate their lead in user-friendly design, but I'd suggest to approach this task with more thought than simply following an existing design.

@ulysses4ever thank you for your attention and remarks.

I follow simple goals with the project:

  • Attract more non-academia newcomers to Haskell. Or at least stop to scare them away.
    Unfortunately, most Haskell-related sites on the web unlikely can be called attractive. It's not very big work to improve the situation compared to other work in the community.

    Disclaimer: I'm not a professional visual designer or promoter. 🤣

    Regarding big bold numbers - maybe it's a bit sad to admit, but these marketing tricks work.

  • I want to familiarize myself with the Haskell ecosystem and community more closely.

  • Make usability better for regular Haskell users. I take all сomments into account and promise that will not sacrifice UX for the sake of something visual.

Copying parts of the crates.io home page is more like the easiest to start a project. Now I have some thoughts and a quite clear plan on what to do next.

I was reading through the original RFC issue thinking it was perhaps another stale issue from 2018. I was so surprised to see a person take the initiative only a few days ago, and it's brought a really big smile to my face seeing this come to life with vibrancy.

As I'm not knowledged on Hackage's web APIs nor am I a professional web developer, I can't do much but to follow this issue with admiration, but I wish you the best of luck and will cheer for you, hoping this design hits completion! It looks absolutely solid and it'll be such a great improvement to Hackage. 💜

Edit: I love the Hoogle integration and the slick search. For the big bold numbers, it may be better to write them as "34.5 Billion packages published" (or the equivalent with e.g. thousand/million), instead of the actual detailed count, because too much detail draws the user away from the impact.

Edit 2: On a closer note, if the real number is only 16,288, the above change might be for the worse. Perhaps keeping the actual number is better.

Day 4.

Started package page implementation.

Screen.Recording.2022-01-03.at.12.07.33.PM.mov

Now I parse HTML content response from proxied HTTP request to hackage.haskell.org.
There are missing JSON APIs. It won't be hard to replace it with JSON APIs after it will be implemented on the hackage-server side.
It will be easier after merging this PR: haskell/hackage-server#996

Now it seems possible to sync HTML parsing with hackage-server without relying on HTML layout and styles of the content generated by haddock.

Will see.

This is very exciting, thank you!

Day 5.

Not so much done on features.

Now you can follow the project at Twitter: https://twitter.com/HackageUI

Day 6.

  • Added favicon, page titles, meta for SEO.

  • Added some links to the footer.

Screen Shot 2022-01-05 at 7 19 28 PM

  • Adapt pages for mobile devices. Menu navigation is missing yet.
mobile.mp4
  • Add toasts for copy package name feature and error toasts for failed requests.
Screen.Recording.2022-01-05.at.2.15.44.PM.mov
  • Search progress indication.
Screen.Recording.2022-01-05.at.2.11.42.PM.mov
  • Fixed several search input bugs. 🐞

Still Day 6. 😄

  • Implemented the "Recent searches" feature. The search history is stored in the browser's local storage.
Screen.Recording.2022-01-05.at.10.40.30.PM.mov
  • Now you can Show help by typing :? into the search bar.
Screen.Recording.2022-01-05.at.10.40.04.PM.mov

Hey! This looks amazing, nice work!

Couple of ideas that came to mind:

  • The search could maybe somehow indicate if there are no results for the search. Currently the user might think that it's still searching even though the search has been completed with no results.
  • It would be nice if you could navigate the search results with arrow keys and enter.

These might already be on your radar :)

@ollimandoliini thank you!

The search could maybe somehow indicate if there are no results for the search.

Right now working on it.

It would be nice if you could navigate the search results with arrow keys and enter.

Yeah. I thought about it. Added it to todos list.

Day 7. Before the second cup of coffee. ☕️

  • Search results header for Hackage and Hoogle searches.

Screen Shot 2022-01-06 at 1 58 55 PM

Screen Shot 2022-01-06 at 1 58 43 PM

  • Ability to clean up search history.
Screen.Recording.2022-01-06.at.1.56.51.PM.mov
  • Disable ligatures by default. Will make them optional later.

  • No results indication as @ollimandoliini asked.

Screen.Recording.2022-01-06.at.1.53.17.PM.mov

Day 7. After the second cup of coffee ☕️

  • Not you can expand/collapse search results items in Hoogle search as asked by @bgamari in #3
Screen.Recording.2022-01-06.at.6.20.31.PM.mov

Day 8.

  • Homepage and Repository info at the package page.

Screen Shot 2022-01-07 at 4 29 18 PM

I think this is absolutely awesome! Thanks for you work, @visortelle. Happy to help with Haskell side of things, if needed.

Thank you, @Bodigrim. 🙂

I have these candidates to solve:

haskell/hackage-server#986
haskell/haddock#1420
haskell/haddock#352

I think @gbaz and @Kleidukos may suggest other prioritized issues to solve that may be potentially useful for the project.


For two or three weeks, I have a plan to make the service usable as much as possible.
Without touching the code of any projects that it's consuming as a data source. I mean Hackage-server, Haddock, and Hoogle at this time.

After that, it would be time to take a break to rationally observe what we got, collect feedback, and plan what to do next.

There are many ideas I keep in mind, but there isn't much sense to discuss them until the initial part of the project isn't finished.

This weekend I plan to finally touch the packages docs part. I'm very curious about what I could achieve here. 😅

Day 8.

  • Display the date when the package has been published last time.
Screen.Recording.2022-01-07.at.11.25.10.PM.mov

Day 9.

  • Fix the "Browse on GitHub" button for git@github.com:owner/repo case.
  • Improve typography of lists and tables on the package page.

Screen Shot 2022-01-08 at 10 09 32 AM

Black background for code with white background for everything else isn't great for my eyes, personally. I hope this either will be possible to change on the user side or will be changed altogether.

@ulysses4ever yep, it will be possible to change on the user side.

Day 10.

I understand it may be controversial for some users, but it's implemented for a purpose.
To evolve the project in the best way, we need to know which features are used and how.

No data collection for ads: https://github.com/visortelle/hackage-ui/blob/83589702c8dd829931ff10a8c83fd6f53d923e30/hackage-ui/pages/_document.tsx?_pjax=%23js-repo-pjax-container%2C%20div%5Bitemtype%3D%22http%3A%2F%2Fschema.org%2FSoftwareSourceCode%22%5D%20main%2C%20%5Bdata-pjax-container%5D#L21

In the nearest future, I'll add the ability for users to hide an IP address or completely opt-out analytics.

More than half of the Internet, including the haskell.org website, uses Google Analytics and nobody really cares. 🙂

Day 10.

  • Make search input clickable, such that one can look them up without typing.
  • Rethinking home page.
Screen.Recording.2022-01-09.at.10.38.07.PM.mov

Day 11. 🙂

Nothing to show today.

  • Working on extracting the search input to a separated JS library.
    It should allow to implement a Chrome extension and maybe a VSCode plugin.

    Will it be convenient or not, will see after release. 🤞

Day 12.

  • Continue work on the browser extension.

Unlikely will release it today, several unexpected issues figured out. JS build stuff is still disgusting if you need something not too trivial. 😕

The same search input as on the site, but as an extension. Here is how it looks.

On page.

Screen Shot 2022-01-11 at 9 26 25 PM

Popup. Will improve styles later.

Screen Shot 2022-01-11 at 9 29 26 PM

By the way, do you have an opinion on what would be a good default shortcut?

Maybe different on different platforms? I didn't think about it yet, please propose what would be convenient for you. 🙂

The whole Chrome extension idea looks like a huge red herring. Of course, as a volunteer you're entitled to work on whatever you choose. Just saying that it likely lowers the chance to hit the goal of hackage-ui, as I see it (to have a fresh L&F for hackage.haskell.org).

@ulysses4ever, why? 😳

The extension itself is not too much work. The React component is the same as on-site. I'll back to the packages page and docs soon.

@ulysses4ever I'll explain a bit. I personally hate context switching and searching a right button or menu item. I'm a fan of emacs's M-x-like flows. Cmd + Shift + P in vscode, Cmd + Shift + A in JetBrains products, etc. Maybe it's a reason why I see it useful. Lot of devs are same.

I personally think the extension is great. Ctrl+H sounds good to me.

Day 13.

  • The browser extension is merged and published to Chrome and Firefox stores.

    No available to install yet. Awaiting moderator reviews.

If you want to help find some bugs, you can install the extension manually.

  • Download zip file:

haskell-spotlight-v0.0.1.zip

haskell-spotlight-v0.0.2.zip

haskell-spotlight-v0.0.3.zip

haskell-spotlight-v0.0.4.zip

haskell-spotlight-v0.0.5.zip

  • Unzip it.

  • Go to chrome://extensions/

  • Set "Developer mode" on

Screen Shot 2022-01-12 at 7 56 26 PM

  • Click "Load Unpacked" and select the "build" directory.

Screen Shot 2022-01-12 at 7 57 14 PM

If no errors, then it is installed successfully. Go to any web page and press Ctrl + h

Screenshots:

1

2

3

4

5

As Ctrl has the same behaviour as Command on Windows, pressing Ctrl+H on Chrome just opens my Chrome History tab. Perhaps it should map to something else on systems other than Mac, like Alt + H? Or add an options page to customise it.

On another note: maybe it's worth creating a separate issue to track bugs and feature requests for the browser extension?

As Ctrl has the same behaviour as Command on Windows, pressing Ctrl+H on Chrome just opens my Chrome History tab. Perhaps it should map to something else on systems other than Mac, like Alt? Or add an options page to customise it.

Seems like a bad default. 😕

Will implement an ability to rebind shortcut later.

On another note: maybe it's worth creating a separate issue to track bugs and feature requests for the browser extension?

Done. #7

Update for early adopters. 😆 Can anyone please check it on windows, does it work now with Ctrl + h 🙏

haskell-spotlight-v0.0.2.zip

Is there a way to install from ZIP for Firefox?

@Bodigrim

  • unzip the archive
  • Go to about:debugging#/runtime/this-firefox
  • Click "Load Temporary Add-on"
  • Select manifest.json file in the build dir

Updated for Firefox:

haskell-spotlight-v0.0.3.zip
.

Day 14. 🥳

It's finally available in Firefox Store 🦊

https://addons.mozilla.org/en-US/firefox/addon/haskell-spotlight/

Screen Shot 2022-01-13 at 10 16 07 PM

Screen Shot 2022-01-13 at 10 17 02 PM

You can install the Chrome version manually. By following the instructions.

haskell-spotlight-v0.0.4.zip

Share your thoughts and star ⭐️ the repo and if you like it. 😁

Awesome work, I believe this is a really big thing for Haskell and its adoption!
Let me know if I can help in any way.

Day 15.

  • Implementing versions, readme, and changelog tabs on the package page.

  • Figuring out how can the extension for VSCode could be implemented.
    VSCode doesn't allow to display of arbitrary HTML, so an unlikely floating input look can be implemented. I’ll try to get convenient UX by using its WebView API.

    Tomorrow moving 🚌 to apartments in another city, so probably will be no news for 2-3 days.

Day 16 - post factum notes.

Browser Extension

  • Fix hotkey picker bugs. 🐞🐞🐞
    Thanks to @ulysses4ever for reporting bugs. 🙂

Updated versions:

Spotlight Input

  • Fixed memory leak.

Day 17.

  • Working on VSCode extension. Got its initial version up and running.
    Wrapping head around and experimenting with VSCode API.

Day 18.

  • Continue work on the VSCode extension.

    Many build-related pitfalls.
    Also requires a lot of testing on various panels configurations.

    Looks not OK yet, but feels quite convenient.
    Will continue tomorrow.

Screen Shot 2022-01-17 at 11 40 39 PM

Screen Shot 2022-01-17 at 11 51 05 PM

Day 19.

  • Released preview version of the VSCode extension. 🚀

Install: https://marketplace.visualstudio.com/items?itemName=visortelle.haskell-spotlight

Screen Shot 2022-01-18 at 10 18 26 PM

Screen.Recording.2022-01-18.at.9.45.18.PM.mov

If any VSCode users are here, please check out it and report any bugs 🙏.

  • Default hotkey is Alt + H, same as in the browser extension.

    How to change the default hotkey:

    CMD + Shift + P

Screen Shot 2022-01-18 at 9 53 59 PM

Screen Shot 2022-01-18 at 9 53 25 PM

🐞 Known issues:

  • :r search by recent queries doesn't work yet.
  • Missing keyboard navigation.

Please put your feedback here: #11

Day 20.

  • Continue with tabs on the package page.

Screen Shot 2022-01-19 at 7 53 54 PM

Days 21, 22 - I was busy with other activities and didn't get a chance to touch the project. 🙂

Day 23

  • Implemented package versions page.
Screen.Recording.2022-01-22.at.7.41.43.PM.mov

I'd like to add more information on each version. Like when it was by published and by whom. Probably license info. But it would be too many HTTP requests.

Version range calculation left for later. If somebody would like to implement the PVP version range calculation function in JS, I'd be very appreciative. 😆

Day 24.

  • Implemented the reverse dependencies list page. Idea by @Bodigrim
Screen.Recording.2022-01-23.at.11.41.52.AM.mov

The feature uses this resource as a data source: https://packdeps.haskellers.com/reverse/pandoc

Day 25.

  • Implemented package dependencies page.
Screen.Recording.2022-01-24.at.4.19.40.PM.mov

Day 26. Mostly polishing and bug fixing.

  • Add 'View on Hackage' button.
  • Make progress indication more visible.
  • Make items more compact on the package versions, dependencies, and dependents pages.
  • Show 404 page if a package is not found on Hackage #14
  • Add "Unresponsive UI" disclaimer on the dependents page in case of many items on the page.
    I will handle it in a better way later. I know the root cause of the problem.

Screen Shot 2022-01-25 at 9 11 07 PM

Day 27. No code today. Mosty planning.

Day 28.

Screen.Recording.2022-01-27.at.5.02.13.PM.mov

Day 29.

Screen Shot 2022-01-28 at 10 39 13 PM

Bonus. 🤣 Played a bit with illustrations, but my wife said that it would be too much for Haskell community.

Screen Shot 2022-01-28 at 11 23 20 PM

Perhaps it will be more straightforward if the name of the (GH) project was the same as the domain name?

@ulysses4ever you definitely right. 🙂 Probably next time. It's not as straightforward as you might think. One platform doesn't allow more than 15 chars in name, another requires week-long moderation for a name change, etc.

My understanding was that, after completion, this UI would replace Hackage at the existing domain (hackage.haskell.org), with help from the Haskell Foundation to transition, using the common backend API.

Did I misunderstand the goal?

@yutotakano you understood the initial goal right. After some investigation, combining several sub-projects into one seemed like a good idea for me. Let's see what happens and if I was right. 🙂

Day 30.

Day 31.

  • Continue to work on the playground. Tomorrow plan to start the playground web widget.

Day 32.

  • Continue on the playground.

As I want to make it suitable for courses, it should be able to run tests.
A lot of build-related stuff ahead. 😓
Most likely will pick the top 100 packages to support as Rust's playground does.

K8s or a single fat machine with docker is one more item on the agenda. Probably the second one from start. 🙂
Need to think about resource constraints.

Some progress:

Screen.Recording.2022-01-31.at.4.33.34.PM.mov

Day 33.

  • Continue working on the playground.

There is a lot of mechanical coding ahead. Probably the same boring "nothing to show" message will be here for the next 2-3 days. 🙂

Days ☀️34, ☀️35, ☀️36, ☀️37, ☀️38, ☀️39 (6 days, really ⁉️) status report. 😅

  • I'm still working on the playground.

    A lot of moving parts here ⚙️.

    Mostly finished work on API and internal reverse-proxy for a single sandbox.
    Implemented minimal UI.
    Tested ability to build sandboxes with a custom set of dependencies, commands, and checks.

    I like the intermediate result, but I won't set any deadlines. Need to assemble all the parts first.