Programming Diary #22: Boosting organic conversations and reflecting on support for open source development.

remlaps - 5 months ago

Summary

This post describes recent work on developing the Steem Conversation Accelerator (SCA) browser extension to help users stay engaged with relevant conversations happening on the Steem blockchain. The extension is intended to provide notifications when followed accounts make posts, comments, or receive replies, in order to increase the velocity and relevance of discussions on the platform.

Over the past two weeks, the focus has been on resolving technical challenges related to date/time handling, network issues, and display formatting for the extension's activity list. Looking ahead, the next month will involve more testing, debugging, and refining the existing functionality before open-sourcing the project. Future development will explore additional features like notifications for specific threads, tags, and communities; filtering options; and integrating Visibility As A Service.

The post also contains reflections on the utility of the SCA as well as possible strategies for organizing communities in support of Open Source development.

Background

During the last fortnight, I continued working on the Steem Conversation Accelerator (SCA) browser extension. The goal for this browser extension is to let the operator know when relevant conversations are happening on the Steem blockchain. Over the years, I have seen many people remark on the fact that it's sometimes difficult to know when interesting comments are made. As a result, gaps in conversations sometimes last for many hours or days. I believe that these gaps reduce the amount and quality of the chain's organic activity.

Enter the SCA. The idea is to create an open source browser extension that will alert the browser operator when interesting conversations are happening, and let them join in through the front-end of their choice. Current options are: steemit.com, steemitdev.com, SteemPro, and UPVU. As with the Steem Follower Checker and the Steem Curation Extension browser extensions, I'm intending to open source this - hopefully by Labor Day (September 2).

For the first set of activity notifications, I have been working on notifications when the operator's followed accounts make posts or comments, or when they receive replies. This was all described in the previous programming diary post, Programming Diary #21: Entertainment and collaboration. The branch that I'm currently working on is labeled, formatHTML, and the following sections will describe progress during the last two weeks.

Here are the descriptions from my git commits after the previous post:

commit e93664a5738f868df37ab81345a76650cb6125d8

- Debugging intermittent network issues.
- Delete not followed entries from the activity list when the actor's
  account is unfollowed by the observer's account, or when the observer
  account changes.

commit f16f23232fd70c9a0c6956cc80a028b1654df897

- Fixed indentation problems.
- Updated to change background color when the data load completes.

commit 47e715168af07f5e54dfb1f33813b8af08ed1ff2

- Time selections and filtering are getting close, but need
  further testing and refinement.
- Much HTML formmating is still needed.
- Should be no more issue with API race conditions as the
  condenser_api.get_account_history call is used for all times.

commit 27e17cdddb5a0a0065903a41d73af099c7d6e105

Still working on date adjustments

commit 3b6b661821a5bb0c2107ea9a516d7501a7cdc28f

Migrating from last_post in list_accounts to timestamp in
get_account_history.  Still need to add the timestamps to the account
list.

In summary, although the intent was to focus on HTML formatting, that has actually been a small portion of the effort. Activities have landed in 3 areas:

• Date/Time handling
• Network debugging
• Display changes for notification and HTML/CSS activity lists

So, let's look at each of them.

Activity Descriptions

Date / Time handling

As previously described, the main issue that I wanted to address with dates and times was a mismatch between the last_post value in the condenser_api.lookup_accounts output and the displayed timestamps in the condenser_api.get_account_history. It appears that the last_post gets updated in near real-time, but the timestamps in get_account_history lag behind. This meant that I was losing activity if I clicked on it after it was discovered with last_post, but before it had appeared in get_account_history.

To address this problem, I decided to ignore the last_post value and track a "last displayed time" for each followed account, then to notify and report on new transactions when they show up in get_account_history.

Easier said then done - Unfortunately, this meant that I had to add iteration to the search for the last posting activity, which made the notification logic more complex, and I had to pass the data back and forth between the background script and the HTML page where the listing is displayed. It also meant that I had to debug a bunch of issues with timezone handling where JavaScript was either reporting GMT time as local time or was applying an offset adjustment to convert GMT time to display in some future time that hadn't even happened yet.

One remaining issue with this strategy is that the rate limit on the Steemit API servers makes the browser extension unusably slow, so I have only found a couple of API servers where it can currently be used productively.

Network debugging

Two steps forward and one step back - After I had the time/date handling up to it's current state (possibly working, but not well-tested), I started working on some other tasks like cleaning up code that was obsolete or inserted for debugging purposes, and when I did, network functions that were previously working started failing again.

I'm not sure if I sorted out all of the problems, but it's been stable all day today - so... 🤞🤞🤞 fingers crossed. It seems that the problems had to do with the error handling. In certain situations, it seems that my code was "hallucinating" network errors, and bailing out when it did.

Troubleshooting this sort of problem is complicated by the fact that I need to wait for a polling interval to expire and for some relevant activity to happen on the blockchain before I can see whether a change actually fixes the problem.

Display changes for notification and HTML/CSS activity lists

Notification changes

The browser notification has now been changed so that new notifications replace old ones, so there is never more than one notification message in the task bar. I also rephrased the text slightly. Here's what the current notification looks like. If there is more activity during the next polling interval, that notification will be replaced by a new one.

Conveniently, that just happened, so here's the replacement notification:

Changes to the HTML activity list

Here's what the HTML activity list looks like now:

Page 1	Page 2

This incorporates the following changes

1. Removed the gaps when posts/comments/replies are not found.
   • At the time of Programming Diary #21, there was a section under each account to report when posts/comments/replies were not found. Now that I'm fairly confident that all data is received, I have removed those sections.
2. Blue background
   • In the previous post, I observed that loading the activity list can take an intolerably long period of time. This has improved, but the problem remained for the browser operator to know when the page had finished loading. For now, I addressed this problem by changing the page background from white to baby blue when the page finishes loading.
3. Fixing indentation problems
   • There were some minor issues with indentation that have now been fixed.
4. Avoiding the display of "not followed" accounts.
   • In my testing, I frequently switch to using the steemcurator01 account as the observer, since their followed accounts are very busy, so this gives me faster feedback. The problem with this is that when I switched back, their followed accounts were remaining in the activity list. This would also be a problem if someone were to unfollow an account while the browser extension is running.
   • Therefore, I added a step during each iteration to delete any "not followed" accounts from the activity list.

Next up

• During the next four weeks, I don't intend to add any new features. Instead, I'll focus on testing, debugging, code cleanup, and trying to make the activity page look cosmetically nicer.
• Once I'm satisfied with the cosmetics and the correct reporting of the currently intended functionality, I'll open source the repository.
• After that, I'll add the following additional features:
  • Notifications for replies on a conversation that the observer has participated in.
  • Notifications for selected tags/communities
  • Filtering of accounts and/or notification types
  • Displaying the Follower Network Strength and reputation scores of authors who appear in the activity list
  • Incorporating Visibility As A Service into the displayed activity list.

Somehow, I also have to find time to look at Q3 adjustments to the Follower Network Strength calculation for my automated voter and for the Steem Follower Checker.

Reflections

Reflecting on the SCA

The original goal for this browser extension was to increase the velocity and relevance of conversations that happen on the Steem blockchain, but I have noticed that it may also have some welcome side-effects.

1. When people are active in their browsers on other web sites, notifications from the SCA may bring their attention back to the Steem blockchain. Previously, I described this as a virtual analog to gravity in the physical world. In particular, I have noticed that when blog authors embed X (Twitter) posts in their articles, I frequently discover myself back on the Twitter web site, even when I had no intention of visiting there. Similarly, Microsoft Teams notifications frequently "pull" the user back into the Teams application. I see no reason why SCA notifications wouldn't act as the same sort of attractor for the Steem blockchain.
2. While I'm developing the SCA, I have already seen many conversations that I would never have known about, and I've even found top-level posts from outside of my follow feed. It turns out that it's useful for content discovery, even in the rudimentary form that it now holds.
3. One side effect that's maybe not so welcome is that it becomes a lot harder to maintain one's voting power when voting on more comments as well as posts.

Reflecting on the Steem development models

Idle and Misdirected Resources - We have this blockchain that can be used as a communication and collaboration tool, but we're not using it to effectively power Open Source development. Why? How can we do better?

1. We need more people. There's no getting around the fact that we're still not recovered from the brain-drain that happened in 2020. So priority #1 is to attract the developers (as @event-horizon has been doing with her Taking Steem to LinkedIn initiative). But, I'm convinced that's not the whole picture. Steem has advantages, and we need to learn to harness them better.
2. Few development communities have the ability to socialize the cost of development by directing rewards to the developers.
   • In the physical world, it's frequently the case that the system is gamed by privatizing gains and socializing costs, but in the Steem world, socializing costs is not gaming the system. The ideal Steem model is socializing costs and socializing gains through the use of competition and proper management of rewards. The challenge, however, is to harness competition and see that the rewards get directed to projects that benefit the entire ecosystem.
   • We have the Steem Proposal System that is almost entirely untapped.
   • We have posting and beneficiary rewards that could be harnessed better.
   • The blockchain pays out ~89,000 STEEM per day in rewards, $15K at current prices. We could fund some impressive development with 10-20% of that, if it were properly directed. In theory, this would increase the value of the token for a true win-win-win scenario (developers-investors-social participants).
3. Related: few development communities have their own online location for self-organization.
   • We need to learn to make better use of communities in support of development. I talked last time about using a community for governance of a Steem working group. We could also use independent communities in support of open source development.
   • I have considered the following models:
     • Perpetual Steem hackathon: Community participants announce their goals for the next two weeks and post their progress, and gain support from investor-curators who are dedicated to supporting development. Efforts could be on any open source project that benefits the ecosystem: new or existing, coding, documentation, design, testing, etc...
     • Communities organize around projects: For example, I could create a community that's dedicated to working on the SCA.
     • Communities organize around development types: We could have a Steem-javascript community, a Steem-java community, a Steem-python community, a Steem browser extensions community, etc.
     • However, as @danmaruschak pointed out quite some time ago (I'll never find the comment again), all of these models bring us back to item #1. We don't currently have enough active developers to sustain multiple vibrant communities, so we need to figure out how to bootstrap any of these.

Conclusion

The SCA represents my third foray into the world of javascript and browser extensions. This is the first time that I've made use of a CSS style sheet in addition to the HTML and javascript language. With the help of AI assistants like Claude and ChatGPT, I'm starting to learn some of the basics.

The guiding principles here are unchanged from last year. I'm looking to improve the Steem ecosystem and also support true decentralization through the use of Open Source code, decentralized and redundant API nodes, and execution on the operator's end-device without relying on other intermediaries (like a web server).

Recent development focused on resolving technical challenges related to date/time handling, network issues, and activity list formatting. The next steps are expected to include additional testing, debugging, and refining the SCA with its current feature set before open-sourcing it.

While the goal of the SCA is to increase Steem engagement, it also offers potential benefits like pulling attention from other sites to the blockchain and improving content discovery. The post also touched on strategies for fostering Open Source development on Steem, including attracting developers and leveraging community-based models.

---

Thank you for your time and attention.

As a general rule, I up-vote comments that demonstrate "proof of reading".

---

Steve Palmer is an IT professional with three decades of professional experience in data communications and information systems. He holds a bachelor's degree in mathematics, a master's degree in computer science, and a master's degree in information systems and technology management. He has been awarded 3 US patents.

---

^{Pixabay license, source}

Reminder

Visit the /promoted page and #burnsteem25 to support the inflation-fighters who are helping to enable decentralized regulation of Steem token supply growth.