Josh On Design

I’ve been quiet lately, not because I’ve had nothing to say but because I’ve not felt well enough to say it. That changes today. 

I’m starting to get a handle on my adult onset Type 1 Diabetes (T1D for the cool kids), including taking more control of my medication and standardizing my food choices (it’s less what I eat and more about consistency of amount and timing). 

I’m not quite ready to look for a new job but I am ready to look back at my career and find patterns. Which areas of tech have I enjoyed the most? What new areas actually interest me.  So I’m starting this article series to document my personal research on the state of the industry. I hope you’ll join me for the journey, and hopefully we’ll learn some cool stuff along the way.

plain, Going Back in Time

I’ve worked in a bunch of [different areas](https://www.linkedin.com/in/joshmarinacci/) of tech during my 30 years of being a professional software engineer (Holy carp! I’m old!). From UI toolkits to network attached storage, from mobile applications to low latency networking. Though the jobs have been diverse, there are some common themes, especially human computer interaction. This should not be a surprise as Graphics, Visualization and Usability was my specialization in college. Still, it’s interesting that I keep coming back to the area over and over.  It’s surprising that my initial instincts at 18 were actually correct.

Let’s go through some current tech areas that are heavy with HCI and are growing.

plain, Artificial Intelligence

I am not an AI maximalist. I don’t believe AIs are conscious, nor do I think we are anywhere close to AGI. Sam Altman seems to think we are, but he carefully hedged his bets by saying we will have super intelligence within a few thousand days, which is a clever way of saying “at least a decade away”. I personally think it will be several times that, assuming it is possible at all. In either case, it doesn’t affect my job hunt today.

However, intelligence and consciousness are different things. It is entirely possible to have intelligent machines that are not conscious, and that is probably preferred anyway.  I’m not even talking about future tech. We have yet to absorb the possibilities of current LLMs, much less the ones coming down the pipe. Lasers were first built in 1960 and we are still discovering new uses for them.

I feel that the next steps for AI are largely a user interface problem. These things are powerful but flawed. Making them applicable to more situations, and making them actually help humans instead of replacing them, is going to require some deep UI work. This is something I want to work on.  If you are in the AI industry and looking to hire someone on the UX side, give me a call.

I’m going to explore this deeper in its own post.

plain, XR: Augmented and Virtual Reality

Working in the Mixed Reality group at Mozilla for three years was probably the best work experience I’ve ever had. The end of that group was devastating for me. I got to build cool demos, developer tools, write blogs, as well as be on the W3C standard committee to make XR widely available and defend against attacks that could fill many a Black Mirror episode. It was a thrilling position and I really miss it.

Since 2020, when Mozilla dissolved that division (along with essentially every other group outside of the core browser), XR has largely been a disappointing industry. Meta has continued to improve the Quest but it hasn’t broken out of gaming. Microsoft shut down Windows MR. Google abandoned Daydream. And finally Apple’s Vision Pro has been a huge disappointment. While the hardware is incredible, how is it that the best designers in the world can’t come up with better uses for AR than 3D photo galleries and VNC?

Maybe evolving VR into to AR was the wrong approach. Maybe instead we should focus on smart glasses with a heads up display and actual useful functions before trying to nail the real world overlay and occlusion problems. 

I’m cautiously excited to see Google return to the immersive space with Android XR, and Meta’s RayBan smart glasses are far more interesting to me than either of their separate VR or AI initiatives.

More soon in a standalone post.

plain, Embedded Hardware

The capabilities of modern embedded systems amaze me. Anyone can make fairly complex hardware using free circuit layout software, embedded programming languages like CircuitPython, easy to acquire components (I'm looking at you RP2040), and cheap PCB manufacturing services. Common standards like USB-HID make it even better. And yet..

Theoretically the low barrier of entry should trigger a Cambrian explosion of bespoke hardware, but I haven’t seen much of that yet. All keyboards look the same. All phones look the same. Where's the crazy hardware?! Yes “hardware is hard” but it’s a lot easier than it used to be.  Where's my LLM powered micro-robots? How are vacuums and airpods the only new consumer hardware category of the past decade?

I suspect there is a lot interesting things happen that just aren't visible yet. One of my personal product ideas is a LEGO compatible macropad, which I’ve prototyped and am considering bringing to market.  I’m also looking at building a programmer alarm clock, pixel style wall displays, and other products that will improve my industrial design skills. None of these ideas are likely big enough to be my full time job, but they’d make for some interesting side projects.  More on this later.

plain, Desktop Operating Systems

I spent 5 years on the desktop Java UI team at Sun, and at least another five years before that building lots of desktop software. I remember the Gnome vs KDE wars. I eagerly awaited every new release of OSX. Those days are over.  

From a UI point of view desktop/laptop OS dev is dead. All of the effort is targeted at mobile while the desktop OSes have degraded from increasing monetization, API neglect, and centralized control; all at the cost of usability and productivity.  A new OS from scratch could be an order of magnitude more productive with far fewer resources if it was designed around simpler messaging APIs and a database filesystem.

Sadly, while I feel there is desire for a desktop OS that is as usable and productive as, say, early OSX and Windows XP, with modern architectures and hardware; I don’t think there is a way to sell it.  I don’t know how to build a business or open source project that could fund professional development of a new operating system (even if it used the guts of Linux to get started).  I have, however, continued some personal research on this topic which I’ll cover in my deep dive on OS and UX dev. I just wish I could find a way to turn it into an actual job.

plain, No Code / Low Code

AKA: coding for the masses. Once upon a time there was a lot of research and product development in software to let non-programmers be productive with computers. To let normal people create their own solutions.  Hypercard. Visual Basic. For a variety of reasons little progress has been made for several decades. It’s sad to me that spreadsheets are still the best end user programming system we’ve come up with. That Apple's addition of math evaluation to Notes is such big news represents how lackluster things have become.

LLMs may be the game changer here. They provide a fundamentally new way of interacting with computers that I think could let people really solve their own problems. I don’t know how to build a business out of it, but there’s clearly something interesting going on here.

To be clear, I don’t think that all programmers will be unemployed because LLMs can generate code for you.  Code generation is not the answer.  Building a secure online application to collect data is far more complicated than it should be. However, asking an LLM to write the same code that a human programmer would is a sure fire way to get an insecure unmaintainable mess. I suspect the right answer is some new model of computation more amenable to the fuzziness of small bespoke applications, and perhaps a new kind of programming language to go with it.

Again, I don't know how to make a business around this, but there's some interesting kernel to be explored here. Contact me if you'd like to talk about it.

plain, Medical Software

The modern medical software that patients interact with (health record systems, patient portals, etc) is a mess. It is some of the worst software I’ve ever had to use (and sadly I’ve had to use it a lot lately). Unfortunately, I strongly suspect that the root cause is that the medical system itself (at least in America) is a mess. *Software reflects the structure of the organization that built it*.  I’d love to help fix medical software, but faster billing systems won’t fix broken business models. I don’t know what the solution is here but as someone currently wading through a new diagnosis I’d love to help fix it.  Please contact me if you’ve got some medical startup opportunities.

plain, Technical Documentation

Writing technical documentation is still harder than it should be and too locked into proprietary software. Writing good docs was the core of my last job, so I’m familiar with a lot of the problems.  While I love the idea of GitBook, the reality of WYSIWIG editing Markdown for large documentation sets is still buggy and frustrating. I have some product ideas and GUI prototypes here that I want to explore in future posts, so I’ll save it for then.

plain, Next Steps

So where am I going with all of this?

Right now my plan is to explore some of these ideas through a small side company (I have a certificate of existence!), while also continuing to get healthier and look for a new full time position.