View profile

data rules everything around me - Issue #3

data rules everything around me - Issue #3
By chekos • Issue #3 • View online
It’s been really fucking busy lately.
This is the end of my third week officially as a data engineer and it’s been consistently been getting more and more interesting and challenging every day.
The day to day activities are unimportant right now - what really has captured my attention has been the overlap between super nerdy topics like database design and equity. Beyond who is and who is not in your database, how well you design your database - when you work in the social tech sector - is really important.
Without going much into detail, imagine you work for a place that handles sensitive data and you provide crucial services, or connect people to such services, every day. Of course, a major negative event like your website going down or losing data will be a major L you take BUT.. what I’m thinking about is opportunity cost.
What if you are cruising along, stable growth, you’re helping people, every quarter you help more - you’re not going through the roof but you’re consistently growing healthily which means more and more people have access to those crucial services and everyone is happy.
What if, you could be doing 10% more by optimizing some part of your workflow (as a company, not even necessarily on your code)? 10% isn’t too much but these are crucial services so yeah, why not?
What if it was only 5%? 1%? What if it was -3% but it meant you’re moving to modern practices which eventually might lead to lower costs, more growth and what not?
I’m having trouble myself finding what is the best strategy here because in my opinion one person that could have had access to the services but didn’t because we didn’t try harder is one too many.
This example is more straight-forward than what I’ve been thinking, actually. What’s been on my mind lately is this scenario: Your company helps people, you have data on what you’ve been doing but your data is not in great shape to be analyzed for some very important questions (in my opinion) like what are the demographics of the people your company is helping? how many people do you help every week/month/quarter?
Like, imagine the data is there. Like, the data exists. Just, the database wasn’t designed for that so it takes you days or weeks to find an answer that kind of resembles what you’re being asked.
Is that impacting equity?
If it takes you two weeks to answer this type of questions, in my mind, you can only answer 26 of these questions a year. Each question answered leads you to make decisions that impact how you reach/impact people. If more information means better decision-making, which means more positive impact, then having a well-designed database so that you can answer these questions more quickly (and therefore allowing you to answer more questions overall) is an equity issue, right?

i recorded like 3 podcast episodes for pycastR and quail data por @tacosdedatos
i recorded like 3 podcast episodes for pycastR and quail data por @tacosdedatos
I had a chance to finally do some creative work - something my soul’s been craving for a while now. I re-re-re-re-launched the @tacosdedatos podcast now named Quail data por tacos de datos and re-re-launched the original Quail data podcast with Rodolfo Ferro aka @FerroRodolfo (he’s doing some dope shit in AI - you should follow him) now named pycastR like “podcast” but py for python and R for R lmaaaooo
Here are this week’s episodes if you wanna give them a listen (both are in Spanish, of course)
tacos de datasheets - Quail data por tacos de datos 🤓🌮📊 | Podcast on Spotify
el niño en el iceberg - pycastR | Podcast on Spotify
pycastR is actually recorded live on YouTube so here’s that link too.
Lo que ando consumiendo...
Here are some cool things I found on the internet this week:
Developer Hack: Creating a Portable PyPI Server for Offline Access
Data Visualization
Automating Data Quality Checks with Great Expectations
Did you enjoy this issue?
By chekos

data rules everything around me. imaginemos cosas chingonas. #blacklivesmatter. data engineer. #dataviz en español -> @tacosdedatos | smol policy bb

In order to unsubscribe, click here.
If you were forwarded this newsletter and you like it, you can subscribe here.
Powered by Revue