Hey there! This is Dan and Ustin (@uzarubin), and we want to share something cool we've been working on for the past year - an open-source `tail -f` for your data, with a UI. We call it "Streamdal" which is a word salad for streaming systems (because we love them) and DAL or data access layer (because we’re nerds).

Here's the repo: https://github.com/streamdal/streamdal

Here's the site: https://streamdal.com

And here's a live demo: https://demo.streamdal.com (github repo has an explanation of the demo)

— — —

THE PROBLEM

We built this because the current observability tooling is not able to provide real-time insight into the actual data that your software is reading or writing. Meaning that it takes longer to identify issues and longer to resolve them. That’s time, money, and customer satisfaction at stake.

Want to build something in-house? Prepare to deploy a team, spend months of development time, and tons of money bringing it to production. Then be ready to have engineers around to babysit your new monitoring tool instead of working on your product.

— — —

THE BASIC FLOW

So, wtf is a “tail -f for your data”. What we mean is this:

1. We give you an SDK for your language, a server, and a UI.

2. You instrument your code with `StreamdalSDK.Process(yourData)` anytime you read or write data in your app.

3. You deploy your app/service.

4. Go to the provided UI (or run the CLI app) and be able to peek into what your app is reading or writing, like with `tail -f`.

And that's basically it. There's a bunch more functionality in the project but we find this to be the most immediately useful part. Every developer we've shown this to has said "I wish I had this at my gig at $company" - and we feel exactly the same. We are devs and this is what we’ve always wanted, hundreds of times - a way to just quickly look at the data our software is producing in real-time, without having to jump through any hoops.

If you want to learn more about the "why" and the origin of this project - you can read about it here: https://streamdal.com/manifesto

— — —

HOW DOES IT WORK?

The SDK establishes a long-running session with the server (using gRPC) and "listens" for commands that are forwarded to it all the way from the UI -> server -> SDK.

The commands are things like: "show me the data that you are currently consuming", "apply these rules to all data that you produce", "inspect the schema for all data", and so on.

The SDK interprets the command and either executes Wasm-based rules against the data it's processing or if it's a `tail` request - it'll send the data to the server, which will forward it to the UI for display.

The SDK IS part of the critical path but it does not have a dependency on the server. If the server is gone, you won't be able to use the UI or send commands to the SDKs, but that's about it - the SDKs will continue to work and attempt to reconnect to the server behind the scenes.

— — —

TECHNICAL BITS

The project consists of a lot of "buzzwordy" tech: we use gRPC, grpc-Web, protobuf, redis, Wasm, Deno, ReactFlow, and probably a few other things.

The server is written in Go, all of the Wasm is Rust and the UI is Typescript. There are SDKs for Go, Python, and Node. We chose these languages for the SDKs because we've been working in them daily for the past 10+ years.

The reasons for the tech choices are explained in detail here: https://docs.streamdal.com/en/resources-support/open-source/

— — —

LAST PART

OK, that's it. What do you think? Is it useful? Can we answer anything?

- If you like what you're seeing, give our repo a star: https://github.com/streamdal/streamdal

- And If you really like what you're seeing, come talk to us on our discord: https://discord.gg/streamdal

Talk soon!

- Daniel & Ustin

Comments URL: https://news.ycombinator.com/item?id=38086387

Points: 63

# Comments: 10

Article URL: https://www.nature.com/articles/s41565-023-01527-8

Comments URL: https://news.ycombinator.com/item?id=38072012

Points: 12

# Comments: 2

Article URL: https://hms.harvard.edu/news/fourth-us-health-visits-now-delivered-non-physicians

Comments URL: https://news.ycombinator.com/item?id=38058902

Points: 31

# Comments: 8

Article URL: https://lemire.me/blog/2023/10/18/how-many-billions-of-transistors-in-your-iphone-processor/

Comments URL: https://news.ycombinator.com/item?id=38061096

Points: 19

# Comments: 3

Article URL: https://linear.axler.net

Comments URL: https://news.ycombinator.com/item?id=38060159

Points: 42

# Comments: 19

Article URL: https://github.com/fellowgeek/esp32-relay-board

Comments URL: https://news.ycombinator.com/item?id=38051380

Points: 11

# Comments: 3

Article URL: https://commonplace.online/article/have-you-seen-me/

Comments URL: https://news.ycombinator.com/item?id=38047821

Points: 6

# Comments: 1

Article URL: https://www.theverge.com/23935029/microsoft-edge-forced-windows-10-google-chrome-fight

Comments URL: https://news.ycombinator.com/item?id=38041434

Points: 8

# Comments: 0

Article URL: https://airportsim.com/

Comments URL: https://news.ycombinator.com/item?id=38026308

Points: 35

# Comments: 6

Article URL: http://geophile.com/osh/

Comments URL: https://news.ycombinator.com/item?id=37984567

Points: 8

# Comments: 0

Article URL: https://www.npr.org/2023/10/22/1207322878/single-parent-married-good-for-children-inequality

Comments URL: https://news.ycombinator.com/item?id=37975090

Points: 81

# Comments: 137

Article URL: https://voidstarsec.com/hw-hacking-lab/vss-lab-guide

Comments URL: https://news.ycombinator.com/item?id=37977502

Points: 25

# Comments: 11

Article URL: https://thephd.dev/c-undefined-behavior-and-the-sledgehammer-guideline

Comments URL: https://news.ycombinator.com/item?id=37967257

Points: 10

# Comments: 0

Article URL: https://www.exploit.observer/

Comments URL: https://news.ycombinator.com/item?id=37956463

Points: 9

# Comments: 0

Article URL: https://www.theverge.com/2023/10/14/23916966/cs2-counter-strike-2-anti-lag-plus-ban-amd-gpu-radeon-rx-7000

Comments URL: https://news.ycombinator.com/item?id=37917696

Points: 38

# Comments: 26

Article URL: https://without.boats/blog/why-async-rust/

Comments URL: https://news.ycombinator.com/item?id=37891020

Points: 49

# Comments: 12

Article URL: https://www.theoutlawocean.com/investigations/china-the-superpower-of-seafood/a-fleet-prone-to-captive-labor-and-plunder/

Comments URL: https://news.ycombinator.com/item?id=37890649

Points: 9

# Comments: 0

Article URL: https://distrowatch.com/index-mobile.php?newsid=11963

Comments URL: https://news.ycombinator.com/item?id=37890305

Points: 8

# Comments: 0

Hi all, hope you are having a good weekend.

I have been a solo dev / indie hacker for a few (many?) years until recently when I added 2 people to my team (one engineer and one for marketing). Initially when adding them to my team I was kind of relieved that they would solve certain problems for me however after a few weeks I learnt while they do what I ask of them they also create new problems for me and I need to prepare a lot more which leaves less time to work solo. My impulsive thought at first was that maybe I should go back to being solo but soon I realised that I enjoy working solo and don’t really know how to be a manager or how to delegate.

Has anyone here faced something similar? How did you learn to become a manager?

I would really appreciate if you could point me to some good sources books videos courses any material that could give me a good 101 on being a manager and delegating work / using Human Resources, also using positive approach whilst giving feedback. Also, do you have any heuristics you use to measure your effectiveness at delegating?

Any help is appreciated, thanks!

Comments URL: https://news.ycombinator.com/item?id=37880690

Points: 49

# Comments: 23

Article URL: https://www.defined.net/blog/would-we-still-create-nebula-today/

Comments URL: https://news.ycombinator.com/item?id=37871534

Points: 18

# Comments: 5

Hey HN! Launching Kali today and I'm super excited to iterate on this.

I hate scheduling and my gf got sick of managing my calendar for me, so I made an AI do it instead. :)

Please let me know what you think!

Comments URL: https://news.ycombinator.com/item?id=37858812

Points: 6

# Comments: 1

Article URL: https://draculatheme.com

Comments URL: https://news.ycombinator.com/item?id=37860086

Points: 38

# Comments: 13

Article URL: https://wordpress.com/blog/2023/10/11/activitypub/

Comments URL: https://news.ycombinator.com/item?id=37847782

Points: 51

# Comments: 5

Article URL: https://paperswithcode.com/paper/rankvicuna-zero-shot-listwise-document

Comments URL: https://news.ycombinator.com/item?id=37846044

Points: 5

# Comments: 2

Article URL: https://archive.org/developers/internetarchive/cli.html

Comments URL: https://news.ycombinator.com/item?id=37834761

Points: 29

# Comments: 3

Article URL: https://www.polarsignals.com/

Comments URL: https://news.ycombinator.com/item?id=37832216

Points: 34

# Comments: 12

Article URL: https://blog.google/products/android/modern-look/

Comments URL: https://news.ycombinator.com/item?id=37821868

Points: 19

# Comments: 10

Article URL: https://phys.org/news/2023-09-definitively-gulf-stream-weakening.html

Comments URL: https://news.ycombinator.com/item?id=37812847

Points: 33

# Comments: 8

Hi HN,

I have been exploring the Hacker News API [0] a bit and came up with this simple way of exploring a user's past submissions including the comments [1].

It's been fun to dig through some of the old submissions and reading the comments, so I thought I'd share.

https://simonpure.github.io/?user=pg

Caveat: For accounts with a lot of submissions, it may take a while to load.

[0] https://github.com/HackerNews/API [1] https://github.com/simonpure/simonpure.github.io/blob/main/a...

Comments URL: https://news.ycombinator.com/item?id=37804111

Points: 4

# Comments: 2

Article URL: https://www.filfre.net/2023/10/the-last-express/

Comments URL: https://news.ycombinator.com/item?id=37799010

Points: 7

# Comments: 0

Article URL: https://abagames.github.io/joys-of-small-game-development-en/

Comments URL: https://news.ycombinator.com/item?id=37799387

Points: 31

# Comments: 1

Article URL: https://www.hpcwire.com/2023/10/05/how-amd-may-get-across-the-cuda-moat/

Comments URL: https://news.ycombinator.com/item?id=37793635

Points: 33

# Comments: 16

Article URL: https://globalnews.ca/news/10006543/ozempic-stomach-paralysis-ubc-study/

Comments URL: https://news.ycombinator.com/item?id=37793185

Points: 36

# Comments: 38

Article URL: https://www.ghacks.net/2023/10/03/windows-copilots-is-showing-third-party-ads-to-windows-users/

Comments URL: https://news.ycombinator.com/item?id=37781266

Points: 40

# Comments: 12

Article URL: https://www.theguardian.com/technology/2023/oct/03/techscape-geofence-warrants

Comments URL: https://news.ycombinator.com/item?id=37755785

Points: 19

# Comments: 4

Hi! Leporello.js is an interactive functional programming environment designed for pure functional subset of JavaScript. It executes code instantly as you type and displays results next to it. Leporello.js also features an omnipresent debugger. Just position your cursor on any line or select any expression, and immediately see its value. Leporello.js visualizes a dynamic call tree of your program. Thanks to the data immutability in functional programming, it allows you to navigate the call tree both forward and backward, offering a time-travel-like experience. Leporello.js offers the ability to develop HTML5 applications interactively, enabling you to update your code without losing the application's state.

It records an IO trace of your program, which is then transparently replayed during subsequent program executions. This allows you to instantly reexecute your code after making small tweaks, thereby tightening your feedback loop.

Furthermore, Leporello.js can serve as an interactive notebook. You have the flexibility to utilize any JavaScript libraries to visualize your data directly within your code.

For a more detailed walkthrough, please watch the product video. Currently, Leporello.js is available as a free online application that you can try right in your browser. My goal is to build the Leporello.js standalone Electron app and a VSCode plugin, both with TypeScript support. Additionally, I plan to add Node.js support (currently, Leporello.js is only for HTML5 apps). In the VSCode plugin, Leporello.js will sit on top of the built-in TypeScript/JavaScript mode, utilizing its code analysis information to enhance the default VSCode experience with unique Leporello.js features.

I am building Leporello.js as a single independent developer. Leporello.js is funded solely by donations. Support me on Github Sponsors [0] and be the first to gain access to the Leporello.js Visual Studio Code plugin with TypeScript support.

I'll be delighted to answer any questions you may have.

[0] https://github.com/sponsors/leporello-js

Comments URL: https://news.ycombinator.com/item?id=37751041

Points: 16

# Comments: 3

Article URL: https://mainichi.jp/english/articles/20230928/p2a/00m/0na/023000c

Comments URL: https://news.ycombinator.com/item?id=37728237

Points: 12

# Comments: 19

Article URL: https://github.com/usebruno/bruno/discussions/269

Comments URL: https://news.ycombinator.com/item?id=37728154

Points: 21

# Comments: 8