I am not a Beginning Developer

[link—standalone]

I am not a beginning developer, or a beginner developer, or a novice programmer, or any other variation of the term. I am an intermediate developer. This means I have mastered the basic skills required to develop a solution from start to finish, with a wide variety of different toolchains. If Software Engineering were a medieval trade, like blacksmithing, I would have reached the Journeyman stage of my development. Traditionally, the journeyman stage was quite important, as it marked when a tradesmen became eligible for full "employee" status under a master. This meant a few very important things, he was able to be paid for his work and his work now reflected back on his master and the trade guild. It also meant that the tradesmen was able to start working towards their own master status and running a business of their own, if they so desired. Reaching master status often required producing a masterwork of some kind to be judged by the guild, and producing this would require learning from different masters, often resulting in journeyman years being marked with travel to work under different masters. It is this aspect of traveling and learning that I want to specifically discuss with regard to the Software Engineering field.

Software Engineering also has a journeyman stage in professional development, it is just generally referred to as a "Junior Developer Role." Your bog-standard junior developer generally has one main goal: to no longer be a junior developer. They may not realize this is their goal, they may think it's "to make more money", or "to become a better developer" or "to find a specialization." These goals are all good motivators, but generally junior developers want to move past the intermediate stage and into the advanced stage.

There are a few ways that the enterprising junior developer can go about this. The most obvious and probably the best is mentorship. If you are lucky enough to be employed at a company who cares about developing the skills of its green developers and provides you with mentorship, be grateful, because this is far from the norm. Most places see junior engineers as chattel, or worse, interns. These are the kinds of places that will load you up with more work than you can possibly handle, and pay you peanuts. Then when you inevitably fail, place you on PIP for a month and fire you. No growth, no recommendation, nothing. Mentoring is a lost art in the professional world in general, and its death is most evident in the software world.

The next best way is open source development. Open source is a great way to learn anything, if you can get stuck into a project and dig in there. It's probably the closest thing to a journeyman program we have in the tech world, as you can learn from some truly great programmers and engineers that way. However, it can be difficult to actually join an open source project in a meaningful capacity. Many have strange practices that will be unfamiliar to a young zoomer such as myself and my peers, like using mailing lists for one. Worse yet, some open projects are run by people who care more about ideology than technology, and those should be avoided, lest ye risk cancellation (Thankfully, these usually make themselves known very easily, just look for the warning signs). However, once you find a good project it can be a great experience (and helps to build those connections recruiters are always raving about).

The last and worst possible to go about your journeyman stage is self learning. Self learning doesn't work because it is almost impossible to verify that what you are learning is correct, as you don't know what you don't know. This can lead to countless hours of wasted effort, or worse, form bad habits that need to be quashed. This state of affairs isn't really helped by the rise of predatory """elearning""" sites, and the absolute state of bootcamp and startup culture. Most people who are forced to engage in self learning seem to find themselves in an endless cycle of "learn thing X, find out thing X doesn't work, learn thing Y" etc. It's tutorial after tutorial after tutorial.

But the real issue with self learning isn't even that, it's that 90% of all tutorials and elearning is tailored towards the absolute beginner. This does make sense, as beginners are most in the need of learning, but there comes a point where wading through the 300 tutorials on how to initialize a project on framework X to find the one that explains the use of framework X's poorly documented feature Z that manager Y has decided will solve whatever the current issue is.

Admittedly, alot of this comes from the fact that my day job is full-stack web-dev, the systems programming world is somewhat different. I have gained more knowledge from bits and pieces of those books that I have in hundreds of hours of online courses and thread cruising on stackoverflow.

Naturally, I state alot of opinions here, and this sure is one of them. This blog has certainly turned more into a place to post personal screeds more than anything else. I don't really mind, it's nice to do something tangential to programming without actually doing programming. Maybe my unstructured thoughts on this topic will coalesce into something more tangible at some point. But for now, it's just rambling.

-ShockTohp Wed, 31 Mar 2021 16:18:59 -0400

Gamers Rise Up

[link—standalone]

The past few days have been quit eye opening for a lot of people. The meteoric rise of value of the GME stock price has shown many of how made up our financial institutions actually are. It has also shown how little divide there actually is between the various camps of political thought among the normal populace. Before this, I would have said that I have very little in common with your bog-standard political activist type, and if really pressed on my political beliefs I would have made some non-committal statements. What I have come to realize from the last few days, is that I have more in common with the farthest left average Joe than I do any member of the current ruling class. And this is the first time that not only is that true, but now for the first time there is a real chance of getting what I actually want, and what I believe most regular people want, a government that is actually for the people it Governs.

This seems to be the American ideal, what with all the "For the people, by the People" language in our founding document. However, if our government has proven anything the past decade it's that it has no qualms about ignoring the people's will for it's own goals. Whether your a BLM activist, or a member of the Trump's crew who "stormed" the capital, or something more radical who thinks the whole democracy experiment is kind of a meme, the US government has ignored and spat on you. This has to end. Government's should provide for their people, not an elite few who pay for special favors. Both sides of the political isle agree with this, unless they happen to be members of the ultra elite.

If Biden is right about anything, it's that it is time for unity. However, that unity is of the populists versus the political elite. It is time for every gamer, from the most ardent Hitler worshipper, to the most hard line comrade, from anarcho-communists to Mr McNuke himself. We need to come together and rise up against the current corruption. Balkanization can come later, once we populists have claimed land to balkanize

Thu, 28 Jan 2021 19:06:55 -0500

UNIX Utility Recreation, cat: Part One

[link—standalone]

The first utility I decided to recreate was cat. While I am sure that the overwhelming majority of people reading this post are familiar with how cat works, however in the interest of padding the word count I'll explain it anyway. Cat is a shell utility that concatenates the contents of one or more files, and displays that onto stdout, it also works with input from standard in. It is an extremely simple utility, almost as simple as echo. This simplicity is the reason I chose to replicate cat first, as it seemed to be the most approachable of the shell utilities from a design standpoint. It also has quite a few options and flags that while simple, may prove to be an interesting challenge. I will go over those options in the next part, for now let's just focus on the most basic functionality.

In this case, the basic functionality can be broken down like this:

1. Check the argument list
1. If no arguments are encountered, echo stdin
2. If arguments are encountered, move to step 2
2. Parse the arguments by iterating over them
1. if a flag is encountered, process it (in this case "processing" just means skip)
2. if a file name or "-" is encountered, add it to the array of files to concatenate
3. iterate over the list of files to concatenate, add echo stdin when "-" is encountered
4. output the contents of each file concatenated
5. exit

I made a few simplifications to this process in my implementation, as we will see shortly. My implementation gives the same results as GNU cat with no flags, verified by sending the output to a file from each and running diff on the files, but has some issues handling signals, such as CTRL-D, which in GNU cat signals to quit echoing stdin and move to the next file in the list. I hope to resolve this issue before the next stage of this utility, and may have to examine the source of gnu cat earlier than anticipated to ensure I am doing things right. This is a learning exercise after all, so I am not too upset about it if I do. I just don't want to make that a habit, so I'm not just copy pasting what already exists.

Let's get into the meat and potatoes of my attempt at recreating this iconic utility. The first feature that I implemented was simply echoing input from stdin to stdout. This function was trivial, all I did was use fgets into an input buffer. I have a defined buffer size that the input buffer is set as, and that fgets uses to check input. This will fail in an input is received that is larger than the buffer size.

Next, I implemented simple file output. My solution is extremely simple, and is included here so that we may all mock it together.

int simpleCat(const char * filename) {
	FILE *input_fp;
	char line_buffer[BUFFER_SIZE];
	char *stat;
	const int chars_to_read = BUFFER_SIZE;
	input_fp = fopen(filename, "r");
	if (input_fp == NULL) {
		printf("Unable to open file: %s", filename);
		return -127;
	}

	while(1) {
		stat = fgets(line_buffer, chars_to_read, input_fp);
		if (stat == NULL) {
			if ( feof(input_fp) != 0) {
				break;
			} else {
				printf("Unable to read from file: %s", filename);
				return -126;
			}
		}
		fputs(line_buffer, stdout);
	}

	return 0;
}

			

As you can see, this is an extremely brutal solution, all it does is simply open a file, ensure the file was opened correctly, then parse the file line by line and output each line once a good read is confirmed. Then terminates when EOF is reached. I also have some return codes, however this is so simple that there is no real need for them, as it will just exit after printing the error.

Finally we can take a brief look at how I am handling arguments. Again, it is an extremely naive and simplistic solution.

int main(int argc, char* argv[]) {
	if (argc > 1 && strcmp(argv[1], "-") != 0) {
		size_t optind;
		int i = 0;
		for(optind = 1; optind < argc; optind++) {
			if (argv[optind][0] == '-' && strlen(argv[optind]) == 1) {
				echoStandardIn();
			} else if (argv[optind][0] != '-')  {
				int stat = simpleCat((const char*) argv[optind]);
				if (stat != 0) {
					exit(stat);
				}
			}
		}
	}
	else {
		echoStandardIn();
	}
	return 0;
}

			

This most certainly won't win any awards for elegance, however it does get the job done. Now you can see the simplification I made to the functionality stated above. Instead of handling the flags, and shunting the file list into a temporary array, we simple treat argv as the file list, and skip over anything that looks like a flag. This will no work in the second stage, as I will have to know what flags have been sent, and how that affects my output once I reach that step. However, this solution performs as expected in the tests that I have run. I am sure there is something I am missing, however, I am pleased with my results thus far.

Conclusion

This project is going smoothly, but after just this little bit, I have seen some issues as far as validation and testing. Ideally, I would like to have a test suite that looks for things like buffer overflows and other weird issue, as well as automates validating my outputs with those of the GNU utilities. But I have no idea how to write something like that currently. I am going to look at testing code like this and try to recreate them before moving on to the next stage. This early I am comfortable taking breaks to make my life better in the long run. Also the signal issue has made me realize that I might be missing some key knowledge for developing shell utilities like this, so I will also be on a knowledge hunt. If anyone has suggestions for a decent testing framework, or were I can find knowledge about how shell utilities are supposed to talk to the OS as a whole, feel free to contact me and let me know.

Anyway, thank you all for reading and I hope to see you in the next part!

Sat, 29 Aug 2020 13:11:45 -0500

UNIX Util Recreation Intro

[link—standalone]

This is a simple project in which I attempt to recreate the GNU core utilities. In order to keep myself fresh while working from home, I have decided to take a deep dive into the UNIX shell utilities (technically the GNU core utils). I am recreating them in no particular order (read: whatever seems easiest first) to help myself better understand C and this sort of development. The way I am planning on going about this is pretty simple, I pick a utility, run 'man [utility]' and see if I can replicate the functionality myself. There are a few caveats, I am allowing myself to use system libraries, unless the functionality is interesting or novel (read: I passed university, I can write strcmp and I don't feel like doing that every project, an example of interesting functionality would be argument parsing). Also, I am going to do my best to keep things in single modules instead of splitting them up into libraries, this is just so I can get really get a feel for ** all ** the moving parts of each utility (this is why I will be using system libraries for things like strcmp). Finally, everything will be compiled with clang, not GCC. I don't have any particular reason for this other than I like clang.i

Every utility I do will have three stages. The first stage I will implement the simplest version I can of the utility. This means base functionality only, no flags, just whatever the core feature is (example, for the first stage of 'cat' I only do stdin echoing and file output). The second stage I will attempt to implement all of the options and flags of the command. The final stage I will compare my produced code with the source code from the GNU source utils, and write up the differences. Each stage will have its own write up, so that I can reflect on what I did, why, and perhaps correct my mistakes. I hope that this project will be beneficial to me in the long run, allowing me to grow as a programmer, as well as giving me some practical experience before jumping into contributing to open source projects like suckless software, or maybe even eventually contributing to linux kernel code.

Fri, 28 Aug 2020 15:53:13 -0500

Cynical Software

While I was still in school I had the opportunity(?) to participate in a hackathon. For those unfamiliar (and to pad my word count), a hackathon is a coding competition where you or a small team produce some software that is then judged by some people who judge software. Some of these have themes, and some do not, and the one that I was in did not. My team decided that we would address what we perceived as an issue in the IT / System Administration world, namely bloated ticketing tools. We brainstormed the problem and eventually came to the conclusion that the issue with most ticketing software is not the main feature, i.e. ticketing, but the clunky GUI that comes along for the ride. We deduced that the GUI is not made for the sysadmin or programmer or what have you, but the managers and admin staff, people who want to make sure the code monkeys are working. However, the GUI tends to lead to an anti-pattern, where a programmer or sysadmin spends most of there time hopping around in shells, or IDE's, but then having to spend more time logging their actions in the GUI. We decided that to fix this, we would right a lightweight shell utility, that could exist on any machine, and log tickets to the ticket service database, and the clicky-clacky GUI could see them. Thus Terminal Ticket was born. It targeted JIRA, since it is very widely used. We built the thing, and came up with a demo and... Came in dead last.

Now, there are a few reasons for this (other entries were way better, we were all first timers, etc), but the one that I want to focus on today is the fact that it was not cynical enough.

By cynical, I mean the school philosophy, the one the OG mad lad Diogenes was a student of. Cynicism al la Diogenes is the rejection of all things unnecessary in life. Diogenes was an extreme example, living in a barrel in the market, urinating in the street, and allegedly throwing away his only bowl after seeing a small boy drinking from his hands. Our software was not cynical and did a lot of unnecessary things. For one thing, we wrote it python, allegedly to "speed development" which, in a way it did because it allowed us to develop a lot of junk in a short amount of time. Python carries a large amount of extras that must be bundled onto a system, so there goes our lightweight requirement. Python is also pretty slow, not GUI slow but still slow in terms of shell, and this was on top of api access, and a complicated configuration to plug into you enterprise's JIRA service and a hundred other things. It also did more than just opening and closing tickets, a lot more. All of this bloat made our presentation confusing at best, and only one judge really "got it."

So, we lost, but I refused to believe it. I railed against the Academic judges. I thought we had produced the next VIM, and that sysadmins everywhere would bow before our superior product. So I made a kick starter, and made 1 dollar. Then I finally gave up, and left it alone for awhile. But it stayed in the back of my mind. It was not until I read this article, from Suckless Software's website, that my eyes were opened. I felt like Diogenes must have upon seeing the child drinking from his hands. "What a fool I am!" I exclaimed, as I beheld the majesty of the erlang ticket system. One command, everything tracked through git, it was beautiful, it was pure genius. I realized the flaw in Terminal Ticket was that we did too much that was unnecessary. We were not geniuses, we were falling to the enterprise trap of doing everything, and the judges could see it. What we had made was a sucky piece of junk that was only "minimal" because it was on the command line.

The cynics sought to remove unnecessary aspects of life, and to live without them. We thought we were removing unnecessary bloat, because we were working around the GUI. In reality, all we did was drag all the bloat into a command line interface, and it was confusing and slow. In reality, we should have been more cynical and focused on only the feature we needed, instead of trying to integrate every feature of the service we were using.

The point of this embarrassing tale of woe is that if you want to make truly suckless software, you have to be a cynic. Remove what is truly unnecessary, especially if it is handled by existing tools. Minimize, minimize, minimize. If you can not do that, then just opt to make something that sucks but looks flashy, because something that sucks but is pretending to minimal will be dropped in a heart beat by the people and programmers who know what they are doing. To round this story out, I am going to share that script I use to track time on personal projects. I developed it in the spirit of the Erlang Ticket system, although mine is only for a single dev, namely myself.

#!/bin/bash

DIR=$(pwd)
TOOLD="/home/kevin/Projects/Python/ticket"
TFILE=$DIR/tickets
TEMPLATE=$TOOLD/template
TEMP="$TOOLD/.tmp"

open() {
	branch=$(git branch --show-current)
	git checkout ticket_tracking
	read -erp "Ticket Title: " title
	ticketnum=$(grep -o \*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\* tickets | wc -w)
	ticketnum=$(expr $ticketnum / 2)
	ticketnum=$(expr $ticketnum + 1)
	cat $TEMPLATE > $TEMP/tmp
	sed -i "s/&l tide >/$ticketnum/g" "$TEMP/tmp" 
	sed -i "s/< title >/$title/g" "$TEMP/tmp" 
	$EDITOR $TEMP/tmp
	cat $TEMP/tmp > > $TFILE 
	rm $TEMP/*

	git add $TFILE
	git commit -m "Added new ticket with id $ticketnum and title $title"
	git push origin ticket_tracking
	git checkout  $branch
}

open

			

That is it. I run this when I start a new task, and when I finish, I open the ticket file, mark the corresponding ticket number as i closed and commit it to the tracking branch. Then I can easily used the commit history on that branch to track time on each task. It it simple elegant, and fit's in very well with my current work flow. Would it work for anyone else? Probably not. But the point is not for it to be useful to everyone, it's to make my life better, and reduce unnecessary work.