Hacker Newsnew | past | comments | ask | show | jobs | submitlogin
Ask HN: Best solutions for keeping a personal log?
238 points by bnj on Dec 29, 2019 | hide | past | favorite | 135 comments
Lately I've been working toward keeping a logbook of what I've been up to throughout the day. I've been using things.app to record things as completed tasks, which has the advantage of being available on both computer and phone and seamlessly sync'ed up.

But I'm interested in being able to pull in more automation--for example, download a csv of transactions from my bank and integrate purchases into the log, maybe add in entries for emails sent, phone calls made. The goal is to capture information passively enough that I can add context and commentary when I want to without spending more time on it than it's worth. I have an idea that it would be helpful for me to be able to look back and get a long term sense of things that I'm doing and interested about.

Do you know of any tools that might make that easier to accomplish? I struggled to find a general purpose app for the iPhone to write new records to a database, and I thought about using a google form but I don't want to have to manually record the time. I thought about sending emails to a specific address, but I couldn't come up with a simple way to do the kind of purchase integration that I'd like to do, and I have a hunch that the whole thing is really only going to be useable if it ends up in a database or spreadsheet form.

With all the personal data collection going on right now I think it would be interesting to start trying to use similar tactics to bring more of my attention to the decisions I'm making about how to spend my time and focus.



I've taken the odd approach of messaging myself in chat software.. in my case, I have "conversations" with myself in Telegram. I feel like I journal more when it's just text and unstructured, rather than, for instance, a detailed form.

It's not exactly what you are looking for, but has some good properties:

- Always easy to jump in to, since I use TG to chat with people all day long; works on all devices (I need to journal from desktop and mobile)

- Easy to forward in/out notes to/from others

- No formatting woes since it's pretty much open ended. Supports hash tags and search. Can easily find all media and links.

- Telegram already supports tons of different kinds of media files and attachments; can easily snap of a pic of what I'm doing

- Can easily export HTML archive of convo at any time for analysis in other software

With my setup you'd have trouble automatically injecting content from programs, but you could make a Telegram bot that does that work, and just have conversations with it instead of yourself. (Telegram's API is incredibly simple to get started with and doesn't even require webhooks)


Exactly for this purpose - using TG as a "personal logging/journaling" platform - I've long thought of building a bot. Since Telegram's bot API is very robust, simple to use and set up, it can be done easily.

I've imagined the bot to:

* save each message in the chat as a separate "log" in a text file that is saved on an online backup. This can be a private GitHub repo, OneDrive, etc.

* give a short summary/statistics of how much I wrote

* tag message with the hashtags I use

...etc.

I haven't got time to start it, but hopefully I do soon. How would you like this sort of a personal bot?


I'd love to check it out. I've wanted to write something similar.

In the past I played with exporting the log and then drawing a word cloud (by week) with Mathematica. It gives you some interesting insights into whats going on in your life in that period.

Other cool features that would be nice: structure extraction (i.e., msg myself `#food #lunch cheeseburger` and `#mood tired` and then building a CSV of only what I ate vs how I felt), reminders (to encourage you to journal every so often, or to quickly have the bot remind you to check on something 12 hours from now), automatically posting external stuff like uptime alerts, calendar entries or commit logs, and other stuff I can't think of right now.. :)


Oh! A friend of mine made a bot like that! https://github.com/martinthenext/daytobase


Yup, I remember taking a look at this bot and its code! It's great work put in by your friend, cheers to him :D

Though I really wanted my logs to be logged in a plaintext file, I'll give this a try again. Thanks!


Maybe related: I have been looking for a way to EASILY record voice notes in a geotagged and timestamped format. Easily here means with the touch of a button, anytime. I think an app by itself can not achieve this, because it requires too much attention. Ideally I would want to use a physical device remotely connected to the phone. Something like this:

https://www.aina-wireless.com/shop/aina-ptt-voice-responder/

But it's use case is push-to-talk messaging and not voice recording. But combined with the right app perhaps... Maybe I should look into chat apps like Telegram?


I'm pretty sure Telegram doesn't geotag voice notes, but I think bots can ask for your location, so it could work.


not sure if this is what you mean, but Voice2Mail lets you send email with mp3 attached and (I think) geotag.

https://play.google.com/store/apps/details?id=com.abelssoft....

https://apps.apple.com/us/app/voice2mail/id533391644

a big ask but would love for this to do speech to text also.


This is great! I was using something similar before creating ZenJournal. Alternatively, you can also:

- Create a private Twitter account

- Private Slack community/channel

- Private Keybase account (E2E encryption)

There are drawbacks with all these though because they are not specifically designed for logging but for fast social interaction over the web, it affects things like UI, security, readability, and searchability, the advantage is they already have a load of features as you outlined. A journaling app could learn a lot from the design patterns of low-friction social apps though.


- Private E-Mail account


I use Telegram the saved messages thread, which I have pinned to the top. You basically send messages to yourself.


I didn't think of using telegram, and that has easy arbitrary file attachments to go with it. I wonder how easy it is to grab the transcripts and transform them, I'll have to check it out!


It's really easy. In the chat context menu, go to Export Chat History. You can then pick what kind of media to export (aside from just the messages themselves). It produces semantically tagged HTML (mostly divs with .date, .from_name, .text, etc, wrapped in a basic head/body tag) in a folder of your choosing which you can then parse with Cheerio, BeautifulSoup, etc.


Another option is the MTProto API, it's pretty powerful, skip the middleman xD


How you find information. It looks like good for 2-3 days or week but after a month you could not able to find the information.


I'm using a similar approach, just instead chat/forward messages to a special private group (I think 'Saved' didn't exist when I set this up).

Then I've got a script (https://github.com/karlicoss/telegram2org) that checks for new messages in this group every hour or so (cron) and appends Org-mode formatted messages to a special file. There are some extra features, like automatically mapping select senders into org-mode tags (yes, I do keep tags for closest friends :D ).

That makes it always show up when I search something (adressing the problem of finding information), I also can easily transform it into a proper todo/task if necessary.


Using hashtags to tag messages can be helpful. Overall, Telegram has a nice search algorithm that plays well with hashtags too.


Very interested in this as well. I use Notion for more in depth notes but I also want a very low friction way to stream thoughts out of my mind and into an organized place.

I built a prototype that does this. It’s got a UI very similar to twitter but organized by day. Special hash tags on a messages at the end of one day appear front and center on the next day so you can get cached in on what you were doing.

I gave up after a few weeks. I don’t use it enough for it to be useful.

So I need to iterate a lot for this to be valuable. I wanted to prototype this before going crazy about building a tool that I wouldn’t even use. Glad I did because now I know the magic needs to happen on the data entry side to make that more frequent.

Ive got a bazillion ideas on how to extract meaning and apply organization to the random little messages ... but without having a personal corpus of entries to mine it’s impossible.

I keep seeing this question pop up though so I know it’s a problem for more than just me, and there is a need in the market. An iOS/Android app was showcased here recently called ZenJournal but it’s not quite what I’m looking for either.

I almost need a lapel mic that can be activated very easily - so I can dictate to this thing as stuff comes to mind. Sometimes the friction comes from trying to organize my thoughts before I write them, which defeats the point.


I'm not an organised note-taker, except for a few circumstances. Most often notes are just actions which get put in email to myself, or my team.

Life has changed recently and I've been thinking about some low-friction options for capturing thoughts. Any solution needs to work from mobile and desktop (macOS, Windows, and Linux for me.)

On desktop I'm imagining something like an Alfred window popping up on a keyboard shortcut. Storing the text note or screenshot with timestamp. Nothing precludes audio, or any other format, but the capture tool needs to load immediately, capture whatever, and disappear into the background. It'd then bug me at specific times to group these notes or add some context.

I'd be able to "zoom" by hour, day, week. It would work out what was during work hours and what was outside of work time, but I'd still be able to indicate personal or work. Tagging would be available, but not required.

I could take some note, or a group of notes "forward" to another tool. It'd store a link to that next system (Trello, work bug tracker, etc.) so I'd know the items were "actioned" in some way.

I'm not aware of any tool which really does this. Note-taking tools are developed for folks who love creating notes and not for people who don't love doing this (or have had excellent memories up to now and skipped development a love for taking a note on everything!)


Your thinking is exactly in-line with mine: ultra low friction, but bug me later to fill in missing details if possible. Also, 'zooming' into or out of a time context is spot on as well. I would love to emit tiny little atomic bursts of thought all day and then later in the evening when I am winding down I could enter a mode where I am organizing/enriching/reconciling stuff. This is an action item, this is a bullshit note I can delete, this was my mood, etc...

Thanks for taking the time to spill out your thoughts on this.


For years my go to option was nvAlt[^0] with a hotkey and some expansions set up with typinator to do all the repetitive stuff. Unfortunately that broke down for me when upgrades to macOS started to make it less reliable and I was transitioning more and more toward entry from mobile. I haven’t been able to come up with a solid workflow to replace that and it’s a pain because it worked so well for a long time.

[0]: https://brettterpstra.com/projects/nvalt/


You might like my https://histre.com/ It (optionally) connects to Telegram, so you don't even have to install an app to quickly save your thoughts. They all go into a "notebook", ready to be organized later (if you so choose).


This is really interesting! I’m really glad you mentioned the telegram connection, looking forward to digging further into it


Thanks. I'd love to hear your thoughts. My email is in my profile.


> need a lapel mic

I saw the Myle [0] at CES 3 years ago, but it ended up never shipping - they took ~$150k in orders but never fulfilled any, sadly. It was a great idea, though.

[0] https://www.indiegogo.com/projects/myle-tap-wearable-thought...


Somebody else already mentioned it, but emacs org-mode is what I use. I have a document in my dropbox with the cheeky name "life.org", in which I contain everything from my sleep log to my recipes to my work lab notebook.

If you're like me, you have trouble sticking to journals/logs. I like having one collapsible document in outline form. I can remember it, and I come back to it. It works well with the natural chaos of life. And it feels more satisfying to add to one big log.


I couldn’t agree more. I think, for me, making any system stick has to include a visible form of progress—if it’s like a blank page every time then it’s so disposable. I hope to have a way to make sure that whether I’m logging using a phone or from a computer there’s a fairly seamless process to sync everything up.

Org-mode is so powerful that it’s hard to discount.


And with orgzly on mobile it works like a charm, I manage the synchronisation with git (one advantage of using plain text)


I'm using org mode (in particular, org capture) on my desktop and Orgzly on Android for active logging.

For passive logging, I've got Orger, set of scripts that are pulling data from different APIs and render it as org-mode [1]. Having it as org-mode allows me for quick search and recall in the logs.

There is also Memacs that uses similar approach [2].

1. https://beepb00p.xyz/orger.html

2. https://github.com/novoid/Memacs


The idea of pushing formats back to org mode is brilliant design.

You basically coerce all manner of new formats into soemthing you can easily manipulate in the future - which is kind of the essential feature of a note taking system.

You will obviously have to update those scripts a little as these other formats change but then again you’ll only have to do so for the systems u really care about.

Would be great if you could wrap all this back up into a emacs package for easy install ;)


Thanks! Yep, you're right about format changes -- that's why I'm thinking hard how to make it as generic as possible and make error handling defensive enough, so it doesn't fail at the slightest opportunity.

Not really planning for emacs package:

- I don't feel very efficient coding in Elisp

- lots of infrastructure required (e.g. scripts for fetching data) is not worth rewriting in Elisp. Actually most of it is quite Emacs-agnosic and it would be a trivial task for Orger to output Markdown/HTML instead of Org-mode if one wants.

- it's a bit more accessible for other people when it's written in Python. I probably wouldn't be wrong if I say that people who are comfortable messing with Emacs are a strict subset of people that can run few Python scripts in cron :)

That said, if you think there are some benefits in having it tighter integrated with Emacs, I'd be really interested to hear!


A few years ago I came to the conclusion that whenever I was struggling with an information task, the correct answer was org-mode. It's remarkable how I forget it over and over again.

I'm using an iphone, do you have any suggestions about how to sync up entries made on the move?


Yep :) It took me several recurring attempts to adopt org-mode as well, but eventually I landed on it.

Don't have iphone, so don't know about the apps. But, even if I didn't have any org-mode specific apps, a tool that merely indexed org-mode files as plaintext and allowed me to search it would already be massively helpful.

For synchronising files across your phone and computer, I'm sure there is Dropbox or Syncthing for Iphone. It's periodic sync, so there is slight possibility of conflicts, but if you use separate files to log on desktop and phone, just merging them now and then, it's not an issue.


Check out organice https://github.com/200ok-ch/organice for mobile


Beorg for mobile.


This is also how I do it. Then I can just occasionally use termux to sync notes up.

That said, in the business world I have enjoyed the many features of evernote, but it's been a few years since I used it.


I use git to sync between desktop and mobile and there is an android client for that called MGit


Not sure why I hadn't thought of that, cool idea. Thanks.


For journals, I use Vim with a ton of snippets, and write in markdown, using a lot of lists. The snippets are the biggest win. I have daily, weekly, monthly, and various ad-hoc templates that I basically just fill in. When I pick up a new journal, I usually begin by writing the templates.

For accounting, I’d recommend taking a look at gnu ledger. I don’t record any financial stuff in my daily journal, it all goes in the ledger repo. The value of snippets applies equally here. I use a crazy set of awk scripts to ingest transaction logs from my bank, etc. but I recorded manually for about 6 months before making them, and I’d recommend anyone else do the same.

I’m not a mobile power user, and have almost no tolerance for typing on my phone (I view it as a read-only device for the most part), so phone-laptop syncing isn’t a requirement for me. I just carve out 15 minutes before bedtime.

Programming your environment is the theme to all of this. IME vim/emacs are the best applications for these kinds of tasks. They’re superior to anything “purpose-built”, because everyone wants to do things slightly differently. They’re more intimate and grind-y to begin when nothing is automated, but it’s trivial to add incremental automation as you get bored of the toil. The early toil is as important as the later automation though, IMO it gives me an appreciation and understanding of the tools I’m later dependent on.


Speaking of vim, vimwiki [1] is a really nice plugin similar to orgmode. It uses markdown-like syntax and has a diary mode. It also supports cross-file and in-file links, tables, formulas in latex, exporting vimwiki pages to html, etc.

[1]:https://github.com/vimwiki/vimwiki


Honestly, I've gone back to paper and pen. It's not searchable, it's not sharable, and it's not elegant.

It is private, it is secure, it is unobtrusive.

My preference is Field Notes https://fieldnotesbrand.com/ and a quick drying gel pen https://www.amazon.com/Zebra-Sarasa-Retractable-Point-12-Cou...


being a lefty i would love to use a gel pen but smearing is an inevitability. even the one second dry time seems to high. i love the field notes . thank you for sharing


Try the Yoropen out. I've been using both the executive pencil and pen for years for longform writing and love it.


Two more low-key ideas here:

https://www.youtube.com/watch?v=Pm9CQn07OjU

Basically, just get into the habit of logging first, then worry about making it seamless.

https://www.artofmanliness.com/articles/the-man-book-creatin...

Go for just the highlights, ignore the noise.

Personally, I tried logging all my finances, my workouts, my diet, my car's miles, etc. for a few years. I never really could get into a very good habit of it, despite any amount of discipline/self-hatred. In the end, these partial and incomplete lists of things became more of a list of 'failures' than anything, personally. What data I could gleam from it was either obvious ('I feel bad when it's been 12+ hours since when I last ate') or they were just too noisy to really get any actionable response from.

Going for the highlights, and on an irregular schedule, was better for me personally.


To be honest I think you’re correct here about focusing on the highlights. I’ve read about things like quantifiedself and I’m honestly really interested in those types of things, but more from a distance—I think too much persistent active logging drags me down and feels like a weight so it isn’t worth it.

When I think about logging, I think about wanting to grab those highlights, and while I’m at it to also unify some of the other logging and tracking that I’m already passively keeping, like my bank/credit card statement. I’ve come around to the idea that I need the smallest number of ongoing projects as possible so that they can continue for the long term, and unifying some of that noise brings so much context if there’s a way I can do it fairly effortlessly.

I have this image in my head of having something in a scrollable/ queryable/ filterable/ or outline view and being able to dive into it . For a long time now I’ve sort of spiraled on the same topics of interest that I periodically return to so I need a log or something to help me expand my attention span to cope with cycles on the order of months or years instead of just days and weeks.


This is my solution but it may not be best for everyone.

Notes:

    # .profile
    function note { mg +-1 "/home/.../notes/$(date "+%Y-%m-%d").txt"; }
mg is micro emacs, +-1 opens the file with the cursor at the end.

Transactions:

I record my transactions into ledger[0]. I wrote an auto-complete wrapper in Python to help speed up entry and another to keep the formatting consistent, but otherwise it's a manual job. It helps keep me aware of my transactions, plus I like to add comments and customize the entries instead of just loading them from the institution's CSVs.

[0] https://www.ledger-cli.org


This one liner is a winner. I have more primitive approach with a single file.

I've been doing it for about 20 years. Services come and go and you can deal with that. But what's crucial to me, is that it is fast. Always. No logging in, just a keyboard shortcut and I'm writing. Waiting for something to load everywhere can be really annoying.


an alternative:

    # .profile
    note () {
      echo $@ > ~/Backup/Notes/$( date '+%F_%H:%M:%S' ).txt;
    }
usage example: ~$ note this is my new short note


I use a similar function in my bashrc. I keep notes on a daily basis with :

  function note { 
     if [[ -z $1 ]]; then
        vim ~/my_notes/$(date '+%F').md;
     else
        echo -e $@ >> ~/my_notes/$(date '+%F').md;
     fi
  }


that's much better


I created ZenJournal for this exact reason. I've used it continuously for the last year and a half, in contrast to all the dozen other apps I've tried. It was actually showcased here a few days ago: https://news.ycombinator.com/item?id=21876939

However, my focus is more manually inputted private thoughts, ideas, and todos, not an information dump with all my activities like what you described.

The app is essentially saving an array of logs with timestamps, so technically it should be easy to merge and blend it with your other automated logs from other devices, but currently, it's kept intentionally offline only with no syncing function.

I found the biggest thing for me is speed to log, ZenJournal accomplishes this with a super quick open and log interface, and also provides a Discreet Mode feature that will help you log in public spaces.

It's also very fast to search because you can simply type ? to start searching.

For those looking for a cross-platform alternative, I'd recommend a private Twitter account too: - It supports most media types. - It exports. - There are many clients and ways to automate it.

It's unnecessarily online to me and the information density is too low with the redundant profile picture and revert-chronological timeline, which is not easiest to read (a private Slack channel or Telegram chat will fix this).


I wrote a little personal messenger bot and it's amazing. I wanted it to be slightly numerical, mostly text, and easy for me to customize and access. So the messenger bot mostly is a wrapper over pushing to different Google sheets or docs (depending on if it's a daily diary, logging exercise/headache/etc)

Having a customized not that's always on my phone has been surprisingly helpful


Awesome! I've been wanting to do this.


Self promotion post, however I'm building an app (https://bustl-app.com) for this purpose.

The application allows users to create projects with custom tasks / notes to record data. These tasks can either include modules to store simple data (like text, task lists, etc) or integrate with NodeJs APIs (Google Drive, Calendar, etc).

I've tried to build it like a simple CMS allowing users to completely customise the tasks and notes they need.

In addition to customising the tasks, you can schedule these to be created at a later date either as a once off or recurring job.


This might get buried, but have you considered using your calendar? I create calendar events in the past to keep track of where I've spent my time. If you want to create an event quickly, there are tons of integrations. You can even use Siri or the Google search box.

I don't do thi s, but if you want to attach things to your entries in retrospect, most calendar providers have APIs that let you edit the calendar description. In any case, the ICAL format is easy to parse yourself. And each event has an ID, if you want to keep the augmented data elsewhere.


This is a great idea, and nicely solves some of the risk of finding one particular perfect tool that won’t be around in five years. I’m going to look at the export options and see if I can make the kinds of notes that I have in mind associated with events


https://roamresearch.com/ blowing up on twitter - 2/3 notebook, 1/3 database


Seconding this, though I'm not sure if it's what the OP is asking for as he mentions automation. I've been using roam and while I wasn't planning to use it for daily logging I'm realizing the daily log can serve as an 'inbox' for random thoughts and things I read that aren't quite worthy of writing a separate page for.

I haven't quite figured out if I'll make it my todo-list/gtd solution yet, for that I'm fairly entrenched in kanban/issue-tracker apps like trello. Also wondering how to use it alongside bookmarking tools like pinboard (or if the longer goal is to have better bookmark features).

--- From the OP:

"download a csv of transactions from my bank and integrate purchases into the log"

- Depending on how much time you have, you may want to check out the beancount community / https://plaintextaccounting.org/ . I don't recommend it to people short on time to go through the learning curve, in those cases just use an existing financial app (moneydance, quicken, gnucash, etc.).

"entries for emails sent, phone calls made"

Roam could work for these use cases, see Conor's recent video: https://www.youtube.com/watch?v=YcNW-eidDJk


+1 for Roam Research, as far as a daily log goes it's wonderful.

> I haven't quite figured out if I'll make it my todo-list/gtd solution yet, for that I'm fairly entrenched in kanban/issue-tracker apps like trello.

I believe both will have to tackled sufficiently before Roam really takes off.

For example, I'd love to be able to:

* Filter for TODO items between $start_date and $end_date

* Filter for TODO items ranked by number of connections (further arbitrary weighting would be really interesting here)

* Filter for TODO items at the intersection of two terms (haven't found an elegant way to do this yet)

> Also wondering how to use it alongside bookmarking tools like pinboard (or if the longer goal is to have better bookmark features).

I'd love to be able to see all the stuff I want to read for #term - this might include books, articles, etc.

Which brings us to content: I really want to read all of this stuff inside of Roam, but there are two approaches right now, neither of which are super comfortable:

1. Copy the whole thing into Roam and process it chunk by chunk. Nice but formatting is a gigantic pain, front-loads a lot of work. Benefit is it saves the content in case it disappears. No easy way to see only the highlights from a certain article as they're all mixed in.

2. Copy metadata and highlights/annotations/quotes into Roam after reading it and creating them elsewhere. More compact but annoying to access original content. Requires yet another tool.


I find ledger's timeclock format [1] to be helpful for tracking my time. I wrote a simple CLI tool to log into and out of nested accounts, effectively billing my time to different categories. I like this because it's simple, human-readable, but structured to support lots of post-hoc analysis.

[1] https://hledger.org/timeclock.html


The Signal private messaging app has a built-in dedicated "Note to Self" contact which you can text, send photos to, send voice notes to, or share anything with via your mobile OS of choice.

I should have added: The idea is that you use Signal only for quick low friction capturing.

In my case, I later process these messages during my daily input review, at which point they mostly end up somewhere in my Emacs orgmode-based system.


All of my banking transactions get put into a sqlite database, imported from csv monthly, and I have a python command line utility that lets me categorize and add notes to each transaction. I have a new sqlite file for every year. I have a number of SQL scripts that do most of the work. The amounts I need for tax returns are generated by SQL queries.

Most of my general note taking is done in emacs org-mode.


SQLite is an underrated interface for this kind of stuff.

Do you support things like nested categories (Expenses:Food:Groceries) and budgets? If so, do you keep them in your data model, or do you construct them in the queries?

Also, just curious, have you considered using gnu Ledger? If so, what prompted you to choose SQLite over it?


No nested categories. Mostly it’s centred around having the correct amounts for tax returns. Some categories like groceries and eating out I have but I haven’t really used it for budgeting—just curiosity. I tag everything as business or personal. I have a Tax/HST column to make that easier.

Every month I get all of my various accounts exported as csv and then import into sqlite with a single command. It was just something I was tinkering with at first and then it became something I used.

I have considered gnu Ledger but my own personal tool does what I need and it was fun to do. So I haven’t felt the need to.


I'd love to see what your schema looks like, assuming it's beyond what you mentioned (and obvious items like date and transaction amount).


My .schema is:

CREATE TABLE account (transaction_date text, amount integer, hst integer, subtotal integer, description text, debitcredit integer, category_id integer, account_id integer, business_percent integer, notes text);

CREATE TABLE category(category_id int, name text);


I switched to Joplin, and it does provides cloud syncing, so working on different devices? It is free and open source!

https://joplinapp.org/


I think a to-do list of some kind would be a good place to start.

I personally have been using Dynalist to keep track of of my daily log. With it, I am able to tell you what I roughly did anytime in the last 2+ years(on a working day).

Here's my current list for 2019: https://imgur.com/a/ZDy4Ydv


My personal log is on Google Docs. It's simple and available on all my devices.

As for automation, I wonder if it might be worth looking at some simple Python scripts running somewhere (a cheap DigitalOcean $5/mth droplet?) pulling data as needed and writing to Google Docs via the Google Docs API? It does require some programming know-how however.


May I suggest my webapp https://histre.com/

You can use it to automatically keep track of your online research, notes etc, and have a nice knowledge base as a result. It connects to Telegram too, so you can take notes on the go. You don't need to install a new app for that.


I've been using TiddlyWiki https://tiddlywiki.com/ in the node.js-based server mode. But I use it more as a journal than a logbook, and not really for structured data.

What I like about it is that I can combine the journal/log aspect with a general knowledge database and seamlessly interlink the two.

Theoretically, TiddlyWiki is ridiculously customizable, so any kind of automation would be possible. Unfortunately, it has fallen victim pretty badly to the "inner platform effect" - instead of letting people simply use the Javascript that it's built on, customizing is supposed to happen in a template language that is just plain horrible and makes even the most simple things like concatenating strings comically hard when done in a context where that kind of operation wasn't planned for.


I’ve never seen this before! It’s great to see that there’s a strong community around it, I’ll learn more about how it’s built. I think wikis are incredible and it would be great to be able to have something like that set up, I saw that there are options for all kinds of different platforms, which has been the biggest challenge for me so far


Love this question. I've written up part of my strategy for doing this here:

https://davidbieber.com/post/2019-12-29-track-your-life-in-a...

The short version is: (1) I manually enter the activity(s) I do each day in a spreadsheet, one row per day; (2) the spreadsheet automatically figures out how recently I did each category of activity; (3) [bonus] a chat bot uses this data (and much more!) to help me keep my life on track.


Lots of health tracking and life logging apps depending on what you wan tot do. https://github.com/woop/awesome-quantified-self https://www.reddit.com/r/QuantifiedSelf/comments/eibrof/ask_...


I use mysymptoms 's other to type notes. And notepad++ with timestamps.


> The goal is to capture information passively enough

This is my top one issue, because most logging apps require constant manual inputs and commitment, but not all.

Recently, I have been using Journey [0], an app that is helping me in the difficult task to write entries nearly every day. Problem: those are free-form, so it's only a good old journal, not a tracking app for custom metrics.

I have tried using another app to track my food habits [1] but that required way too much effort to get it right (finding the right food for every intake is just too much, I'd rather wait for something more automated like computer vision accurately seeing what I am eating, and how much of it).

I passively track my spendings, cross accounts with an bank aggregator called Bankin [2] (EU focused). From it, I can export the normalized data. You should look into similar services.

> a general purpose app for the iPhone to write new records to a database

I don't like it, but Notion [3] does that.

In the end, if you're committed, a spreadsheet is not that bad to start prototyping and checking what your really need.

Don't forget passive tracking with apps such as Google Fit, Swarm (Foursquare), Google Location, etc.

[0] https://2appstudio.com/journey/

[1] https://lifesum.com/

[2] https://bankin.com/en/home.html

[3] https://www.notion.so/


These are great suggestions, thank you! I think you're probably right that a spreadsheet is a good place to start, perhaps with IFTTT / Zapier / Shortcuts / etc to try to start plugging in some of the automation. I put a custom function in a google sheet that captures the current time when writing in a cell, and that platform has cell-specific edit history now as well.

It's a shame that there isn't a bigger player in the space for user friendly custom metrics like this


I'm a desktop only kind of guy, and have taken to just logging/journaling in markdown via VS Code - which is what I'm most used to for code documentation and the like, so why not reuse it?

It sounds relatively easy to generate a .md table from a .csv file which could have an annotations column for any additional custom notes to be added to if desired. While I haven't done so for logging/journaling, I'm doing basically this for my rust crate reviews/audits, which have the benefit of being public: https://github.com/MaulingMonkey/rust-reviews . That autogeneration code can be found in 'src', but it's throaway gross badwrong code, so don't expect much ;). It pulls metadata from the yml-ish preambles of reviews/*.md, and from `cargo crev` commands.

The .vscode/tasks.json file in the repository means I can just Ctrl+Shift+B to regenerate the index. In a journaling context I could see adding more tasks - perhaps one to, say, create a new journal/log page - pre-populated with whatever auto-injested information you might want.

Returning to my journaling - I keep it as simple as:

    C:\home\notes\journal\old\...
    C:\home\notes\journal\old\2019-05-26.md
    C:\home\notes\journal\old\2019-06-07.md
    C:\home\notes\journal\old\2019-06-13.md
    C:\home\notes\journal\...
    C:\home\notes\journal\2019-12-02.md
    C:\home\notes\journal\2019-12-03.md
    C:\home\notes\journal\2019-12-21.md
    C:\home\notes\journal\2019-12-24.md
With the old folder serving as a dumping ground to keep the root folder managable at being just the most recent 10-30 entries or so. A page generator could easily create pages in a convention that allows stable linking between them:

    ...
    journal\2019\12\21.md
    journal\2019\12\24.md


CSV with some cloud sync solution. I turned the overhead of manually logging an activity into a feature of committing to a task. I log Date, Start Time, End time, 'Project', and location. When I feel myself losing focus I end the task and take a break. Early inspiration from Cal Newport.

Supplement this approach with a small paper notebook (originally I used 3x5inch pocket then moved to a 5x8 inch) where you no longer need your phone/computer to log entries (enter them later). I'm collecting a nice physical library of notebooks that are chronologically ordered and contain everything from in depth technical tasks to notes on groceries. Also acts as a good spot for stickers collected on your travels.

Regarding transactions, I've started using everydollar.com. I've only started using it in 2019 so I cannot 100% vouch for it yet.


> The goal is to capture information passively enough that I can add context and commentary when I want to without spending more time on it than it's worth

If the goal is to pull data from potentially multiple sources, then I think you should think more about how you are going to analyze the data than just how you will just capture it.

I have been extensively tracking my spinal cord rehab for over two years. I use Google Forms for some of the manual daily tracking, but lots of it comes from other sources (Apple data, a bunch of health apps, some spreadsheets, etc).

I use some python and bash scripts to then import it all into Elasticsearch and build dashboards: https://github.com/gibrown/es-health-tracker


That’s a great point, I do need to figure out what my end goal is before I sink a lot of time into having a capture system. I come from a line of librarians and I often find myself falling into information capture and management related puzzles, and while it’s nice to build resources that can go on to fuel unexpected opportunities, it’s equally important to be able to articulate goals for a project like this.

I think part of mine is a general anxiety about memory, wanting to be able to have a passive, external, searchable record that can help me hold onto context that I’ll otherwise forget. I think that having a log which is flexible enough to be the bucket into which everything can be placed will also help me solve a problem of having lots of different places that different types of things end up—it fractures my system of organization in a way that makes everything a little unreliable. Inventing this new log is just another symptom of that.

How I want to use and analyze it is the next thing I need to hold myself accountable for answering.


https://github.com/tomlockwood/dn

I made this, which some people seem to like. Notes sorted by day, from the command line. Also, not much code, the source can be read in a minute or two.


I like this. Thanks. You can combine this with syncthing and have the notes folder synced to multiple devices.


My opinion:

- Boostnote attached to cloud storage or a personal NAS. Which I still don't like much (no mobile support). Or use OneNote Cloud (Or Google Cloud), but if privacy is an issue (like legal notes) then hard pass most likely.

Failed attempts:

- I've left Notion because it's not very exportable for DR and movement purposes.

- I've used Standard Notes which I loved, but it's clunky and very expensive.

- I've also tried the basics like Apple Notes, Evernote.

Overall, it's hard to find a portable solution that can also be stored online or offline with slightly better formatting than markdown. So the best option is use an editor that handles Markdown, since it is very transferable and store it somewhere easy for you. For database related functionality stick to CSV.


I just keep a git repo of raw text files with dates. It's dead simple and I usually just do everything via cli with vim.

Maybe some day I'll get around to developing a simple markup scheme so I can parse and search my logs by #ideas #opinions etc.

I'd use a python tool like jrnl, but vim, text files and git are just too simple to complicate with tools that don't really offer that much more functionality.

I had a less technical friend who used to obsess over software to manage bookmarks, notes, etc and it drove me nuts how over-thought / contrived his approaches were.


> I thought about using a google form but I don't want to have to manually record the time.

Google form entries have an automatic timestamp.

No other ideas besides the ones already stated (also a jrnl.sh user).


  - Google Keep ( chronological, good search, hashtags, simple, fast)
  - Evernote ( more features)
  - MS OneNote — Comparable to Evernote but free. 
  - A github repo + `ack|grep|fzf`
  - Apple Notes (only if you exclusively use iPhone
  - Keybase Files + Encrypted Git repo (an encrypted alternative to github)
  - Moleskine pocket notebooks
My breakdown is 80% keep, 15% moleskin (then I transcribe to keep) and 5% Keybase (for off-the-record journaling)


I used to keep a moleskine in my pocket absolutely everywhere. I have a great stack of them. Being able to sync and search a digital alternative is really attractive, but I do think it’s telling that I’m here asking the community for help essentially reinventing pen and paper.


I use a plain text file, and I edit it in Emacs' fundamental mode. I have a key binding that goes to the end of the file and adds a timestamp and some separator characters, but that's the only customization.

This single text file goes back 30+ years now, and it's survived all the fancy-pants TODO and notetaking software I've tried (a lot of which probably doesn't run any more). I think that long-term habits work best when their tooling is simple.


I've been using Evernote. One feature I really like about it is the web clipper. Doesn't do any form of analytics afaik as it's designed for memos.


I've wanted this for a while. For example, every time I have a conversation with a contractor, CS rep, etc. I wish I could add a quick note about what was discussed and attach any supporting documents.

I want to enter these chronologically, as they happen, but then have them viewable like email threads (like the main GMail view) grouped by business or general topic (like "House Maintenance"), with the most active threads shown first!


I played around a bit with putting it in commit messages on an empty git branch, hanging my current working copy on the commit as a parent.


I’ve been using Day One for a while. I started using it in 2012 as a simple journal but it became a essential tool in logging my life.


I started using DayOne about two years ago and I've really enjoyed it! I like that it does a lot of the work for us. I do want to dig into the privacy policy a lot more, because the "activity feed" feature seems like it would be really useful but obviously I'm concerned about the persistent location tracking.

I've been trying to come up with ways to automate entry of other data as entries in day one, because something that's really useful is that you can grab a copy of the sqlite database underneath it and do what you want with it.


I use timing app https://timingapp.com for exactly this.


For Automation maybe you could try IFTTT to pull data from one service to a google spreadsheet. Since it has dates associated with the data, you can later match them with your current dataset.

I know a service called exist.io that does a quite similar thing based on API. You can check that out if a central hub for all personal information is what you're after!

Cheers!


Built an app a few months ago to capture my stream of conciousness. The UI is like a messaging app where you can just send yourself messages and quickly organize them under tags.

https://www.streeeeam.com

I wonder if this could turn into something like parabola.io but focused on consumers.


Apple Notes. Always with you, and searchable.


I’ve been using notes a lot lately, the document capture is brilliant. I should learn more about it, but I’ve had some trouble figuring out how I would want to export and manipulate the information in notes if I wanted to—it’s seemed like the types of features I would be looking for: export, get everything into a database that I can interact with and manipulate... those are some areas of weakness in notes, it seems. Another strange thing is the difficulty in finding the dates associated with note creation and edits; you can sort notes by that and you can pull down on the note to reveal the date and time, but as far as I’ve found in the past it’s hard to grab all your notes and get the dates with the content.

I think it’s really promising to see how fast notes has been evolving—it seems like with every new release Apple has been adding and polishing to make it a really powerful option.


Second this. If you're in the Apple ecosystem and want something with markdown, I can't vouch enough for Bear https://bear.app/


On a side note - why are bank apis so sht?

Would love a clean read only way to http api my 3 banks. I don't even care if the security is rubbish for read only. Csv imports are bullshit. There is no reason why this can't be live and clean

So hard to keep track of things when it's spread over multiple banks & jurisdictions


I also wonder about this myself, and my conclusion is that it probably increases the bank's systems attack surface for very little gain (in their eyes).

Very few people have the knowledge and interest in interacting with a bank's API, so there's little upside with a lot of downsides. Even if read-only, something like this has to be architected correctly and we all know this is most often not the case.

On the other hand, if businesses are built around these APIs, banks might perceive them as competition and no executive is going to choose to support them.

Even though I (and probably others in HN) can see the value in offering more modern banking features built in top of official APIs, just look at what has happened with e.g. Twitter's API and others. Eroding your own vendor lock-in is generally frowned-upon by most companies, no matter how much we believe it might be win-win for everyone.


If you're in the US, I'd recommend checking out either Mint (by Intuit) or Personal Capital. Both use OAuth for the banks who support it. No handing over of bank credentials to a 3rd party and the sync is near real-time from my own testing of both.

https://mint.intuit.com/ https://www.personalcapital.com/

If you want something with more functionality to do budgeting, You Need A Budget (ynab.com) has a lot more functionality in that department, but does not support OAuth connections to banks; in my test period I had to re-add my bank accounts constantly. Shame, because the functionality of YNAB is stellar overall.


If you don't mind a third party having access to your bank, you can use Plaid. Last I knew they supported free accounts with basic balance + transaction support and the first 100 accounts were free. I've definitely seen some home spun personal finance apps using Plaid for free.


>If you don't mind a third party having access to your bank, you can use Plaid

I'm down with read only access. Full access.. Fk off

But even that is a shit show. Eg a major card provider might push a api... But sorry not in your jurisdiction..not EU so you don't get EU benefits

As best as I can tell not one of the half a dozen cards I carry can do a credible read only api of any sort


Plaid is read only so far as I know. To do write type actions you bring in another party like Stripe.


The problem with your request is timeline. I found more productive is to keep essence of things in a easily searchable format. Timecharting very diversified information is difficult and quite often not needed.

To keep snippets in an instantly searchable format I use Google Keep. This is so simple and perfect.


I don't specifically do time-based logging, but I do have a local Mediawiki instance for knowledge about my life. You could probably easily write a template for doing log entries, so your Mediawiki syntax is short and sweet, but your log entries are nicely formatted and timestamped.


Probably not what you're looking for, but for the extreme data nerds RescueTime provides really interesting insight when viewed across several years. But of course, paying for it has to be driven by a desire to know that kind of information rather than a practical need of any kind.


Sounds like these might be useful to you...

https://mint.com for your spending habits — https://rescuetime.com for your computer habbits (email, web, apps)


I have to ask: Why do this? What’s the motivation? What are you hoping to get out if this exercise?


Used DavidRM's "The Journal" extensively in the past. Is Windows only, but works well for what it's built for, and is well supported.

http://www.davidrm.com/


I use a paper notebook: write notes, date each page, and add entry to the index at book front or end for ease of finding it. I carry it everywhere & and it requires no batteries:)

When notebook fills up I scan each page into Evernote, which OCRs it.

Low tech but it works.


I have a feeling you're overthinking this; a simple spreadsheet will be easiest. Ignore the integration bit for now and just start by tracking your time/tasks. Do it for a week and see what you think.

Personally, at the end of each work day, I use the spreadsheet method to record my progress for that day. I have one column for the date and one for a 1 or 2-sentence summary or notes about what I did during the day. I've been doing this for years and it's nice to have a historic "captain's log" of my work.

It would be easy to add more columns to this type of spreadsheet to break the day into hourly increments or even 15 minute increments. Any shorter than 15 mins and it's too easy to get frustrated by the interruption of needing to write the log. Hourly would probably be more realistic.

In Google Sheets, use command-shift-colon to do a timestamp.


Yeah you’re spot on that starting with something and moving out from there is the best way to not shit myself down.

> In Google Sheets, use command-shift-colon to do a timestamp

Okay not for nothing but I use sheets constantly for work, and as soon as I’m back on a desktop I am stupidly excited to try this out


I use Joplin. Bare bones relatively speaking but I like all the files are markdown


I use Zettlr + git to keep my personal log/lab notes/daily journal. It's just a fancy markdown editor, to it should be trivial to write some shell foo to pull in data from whatever source you like.


I'm using paper notebooks of various kinds and I am happy with that.


Check this :

https://austinkleon.com/tag/logbook/

I started actually something like this for a month now :)


what I use to keep tract of a lot of my data point is exist.io it has a lot of automated import from various sources to collect everything in there and see patterns


I typically just use the default "notes" app on my Neuralink implant. Pretty basic UI but very low-friction for stream-of-consciousness writing.


org-mode, emacs, and a git repo


Org mode has a diary feature so the git repo not strictly necessary.


well, it makes for real simple backups... and you have magit right there just a C-x g away, too.


With git-auto-commit [0], you can define a hook that commits and pushes to a private repo every time you save your note. Set this up after a HDD crash wiped out 6 months worth of notes.

[0] https://github.com/ryuslash/git-auto-commit-mode


I usually just use magit and push at EOD, but i’ll keep this in mind.


I use typora(https://www.typora.io/). It works for me well.


I like Workflowy


Same here!

It is simple, searchable and extremely flexible.

I have a list for each date in a Logbook list and keep dumping all my completed TODOs at work there. It has come in useful a few times already to recall when did I do thing X.

I also have an Inbox list to keep all my thoughts in there, which get groomed once a couple of weeks. The mobile app is a little slower than I'd like but it is the best tools I've found to organize my thoughts.

I also get daily emails from Workflowy which provides a nice timeline of everything I've done or recorded at any point.


Pen, a G2 (my favorite) and a 3x5 spiral notebook.

Things are compiled and distilled when not on the go.


Jrnl.sh might be a good fit.


Thanks for the suggestion! I think one of the big strengths of Jrnl.sh is the simplicity, the end result is useable and straightforward. I wish it could be easier to use a phone to append lines to something like this, I thought about trying to make a shortcut that could do the trick, and then manually transforming and integrating the contents of other sources like transactions periodically. My worry is that manual steps in the process end up making the whole effort too troublesome.


My note and log process is pretty manual, but I've been tinkering with a system in Evernote for the last 5 years and it has started to really pay off over time. My goal is having a clear log of what I did, when I did it, and any useful references all integrated into my general note-taking system, so perhaps some of these observations could be useful:

1. It is a highly personal process. I've tried copying high-level systems (like GTD and its derivatives), specific note-taking / TODO paradigms, productivity/habit tracking apps, and other journaling approaches, and I've always found that if the system is not built around your own habits and preferences, you will eventually lose motivation. Figuring out your own style seems to be important. Another general principle I’ve found is keeping the tools simple. Mostly text-based, always available offline, built to last, etc. Develop the habits, then adopt fancier tools.

2. For quick context-capturing and bookmarking, I rely on email and Pocket. I prefer email because it is reliable, it'll be here in 20 years, and it is fast to flag a message as "needs attention" or e-mail yourself an important thought or reminder. Pocket seems to just work across all my devices, so I haven't had any reason to branch out. Whenever I have time during the day, I organize this “unorganized” view by either taking care of the item (if it’s a quick article in Pocket or errand in email), skipping it (it’s fine to de-prioritize things until later), or by triaging it into Evernote:

3. I use Evernote for a lot of things (including all my technical notes, paper highlights, personal journaling, my "principles" document, philosophy on nutrition, habit aspirations, workout plans, and so on), but of interest for this question is probably a topic-specific log and backlog system. Near the top of my Evernote shortcuts are four log links (Daily Log, Technical Log, Health Log, Workout Log).

4. Daily Log is where I can make a plan for each day, or put nothing here, or just have a place to ramble (writing down thoughts helps internalize them, after all). This is something of an unstructured personal journal.

5. Technical Log is for all technical work that I undertake (coding train-of-thought, reading papers, reading blog-posts, etc.). In reality, under each date, this log contains a bunch of Evernote links to more detailed notes on specific technical topics, but the big benefit here is being able to look back over the course of a week or month about what kind of insights or interesting concepts I’ve come across. It’s particularly useful to read the day after and get most of the context back quickly. I usually always have this log file opened.

6. Health and workout logs track both metrics and how I generally felt that day (health-wise and workout-wise, respectively). It’s ok to have empty days. The important part is filling in on days when things are particularly bad, or particularly good, so you can reflect.

7. After this, I have a bunch of backlog shortcuts — so when I triage items from Pocket or my email, they go into the appropriate backlog list. Examples for me: Backlog of {ML reading, CS reading, Bio reading, Travel destinations, Video/TV/Movies, Restaurants, Games, Running routes, Books, etc.}. Backlogs are great -- they are lightweight, they can grow indefinitely, and you'll naturally prune them of things that you don't care about after some time. Really, lists are a great way of writing things down! When I'm at a coffee shop, I'll pick and item from one of my reading backlogs, if I'm not already working on something. When I have time to plan a weekend, I can scroll through my recreational backlogs and combine some items to fill out my schedule.

8. I like having a high level view of what I did and what I have coming up on long time scales, and for this I use a big spreadsheet, one sheet per year, one row per week, where all past and future social events, travels, and “macro” events. It’s useful to be able to look back at months or years of events and figure out who you spent time with, how often you traveled, or went out, etc. I prefer the spreadsheet to a calendar app because I don’t feel the pressure of adding specific times and I can keep it uncluttered.

Having a centralized place (along with a calendar or some kind of macro-overview) seems to work well for me, because I can easily cross-reference just about any day of the year to figure out what I did that day, my state of mind that day, and so on.


Evernote isn’t perfect, but it works well enough.


A personal slack.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: