You are viewing a read-only archive of the Blogs.Harvard network. Learn more.

The Silo Mentality: Keeping It Up While Breaking It Down

Glass silos

Building glass silos. Making them transparent and permeable.

I’m going to try something new and actually follow up to a previous post that I wrote, oh, nearly four months ago. In the article (yes, I’ll call it an article) “The Silo Mentality: It Can Be A Good Thing If You Don’t Do Your Own Thing,” I argued that the Silo Mentality isn’t inherently evil. As an example, I argued that a small team of software engineers who specialize in developing pedagogical tools are going to have (and need!) different processes and standards than a huge team responsible for enterprise-wide applications. To bring it down to common sense: It’s often necessary for departments to be different–after all, if they weren’t different, then why bother separating the departments in the first place?

Silos only become troublesome when they are truly isolated from each other. Silos that don’t interact start duplicating efforts and start needlessly competing for the same resources. At the same time, they stop sharing information, and cease to take advantage of expertise that certainly exists outside their walls.

So to break down silos without actually tearing down the walls, one article recommends that an organization make those walls transparent and permeable. Transparency implies that individuals outside of the silo can see into it. They can view the people, processes, and products of the silo–get a clear idea of what work being done. This assures outsiders that whatever is happening within the silo is best for the overall organization, and therefore builds confidence and trust, perhaps even respect. Permeability suggests that there is an exchange of information between those within the silo and those outside of it. The benefits of this are–well, the article tends to use the words “leverage”, “optimization”, and “expertise” a lot when what it really means is that a permeable silo gets people talking to each other.

Which brings us to yet another commonsensical point: Remodeling our silos means communication. Yes, that old gem. We’ve all heard it before. And we all know we should be doing it.

Many of us within Harvard University Information Technology (HUIT) have been attempting to make our individual silos more transparent and permeable through various methods of communication. Here’s how:

First, we have the blessing of the organization and its senior leadership team to go about doing so. The overall culture of the organization is based on four core values, two of which are to be open and collaborative. This gives us the freedom and the motivation to reach out to each other.

Our leadership is also sponsoring a series of Open Houses, in which each major area, such as Security, Infrastructure, Academic Technology Services, etc. devotes an afternoon to opening their doors so the rest of the organization can stop by and see what they are doing. These Open Houses have blossomed into actual events where the entire organization gets together to play games, eat, drink, and socialize while, at the same time, learn about the department they are visiting.

For the software development teams, the HUIT Software Development Standardization Project assembled a core group of software engineers and management from all over the organization. Our goal was to create a series of high-level recommendations (and the keyword is “recommendations”) for development teams to consider. This way, we brought the disparate groups together to discuss common issues, and arrive at potential ways to solve those issues. At the same time, we didn’t force any one solution on the teams.

The Standardization Project has continued in various incarnations over the years. Most recently, a core group of developers has been sponsoring a monthly series of events called The Month of Things, where we pick a topic, such as accessibility, and cover it over the course of four weeks, the first week being a brown bag, the second week a workshop, the third week office hours, and the fourth week a free lunch.

Finally, developers from various academic technology teams have been getting together in an informal Community of Practice, wherein we meet monthly over lunch, and one of us presents a topic that might be of interest to the other developers. We also end up sharing ideas, and a collaboration or two has sprung up as a result of the meetings.

Despite all the talking, sharing, and collaboration, we’re all still working in our own silos, and that, as I ave stated before in so many words, is a Good Thing. The important aspect to remember is that the silos have become more permeable and transparent. They are silos made of glass, with many open windows and doors.

 

Posted in Management. Tags: , , . Comments Off on The Silo Mentality: Keeping It Up While Breaking It Down »

Version 1 is a Joke

First off, I am not talking about VersionOne the Scrum management software.

We have been working toward version 1 on a couple projects for a while. The projects are released in pilot mode, and new releases are pushed to production every week or two, but they aren’t at what we are calling “version 1”. Which is what we are thinking is a finished product.

Scope Creep is Terrifying

I think a thriller/horror movie called “Scope Creep” would do well because I think a lot of people would relate to the horror of it. We have, multiple times now, put all of the stories in our backlog that we considered to be “version 1” into a Jira version. And those stories have all been completed. And then we add more. And those are completed. It is an endless cycle because we’re missing the decision making process of we’re there.

The Perfect Product

Show me a piece of perfect software and it will probably be the “ls” command or something. The key to perfection is simplicity.

Trimming the Tail

It is usually the responsibility of the Product Owner to trim the tail. However, you can’t always rely on that. It is the responsibility of the TEAM to prevent scope creep. Everyone has a say, everyone can see it. No one wants to be working on the same project forever.

only you can prevent scope creep

Posted in Uncategorized. Tags: . Comments Off on Version 1 is a Joke »

Offices and Opens and Saving your Work

A thread picks up on reddit every so often about developers needing quiet and the evils of open floor plans.

Many years ago I got a new job and during the interview process I was told I’d get my own office. There were several open offices, the person I was replacing had her own office. It was a sure thing. My first day I was grouped with the most junior developer of the group. I hated it. For a year or so I was the only person who had to share an office. (Yes, I see what I did there, this article is about me, not him.) After a week I was used to it.

Then the dev team moved to a new location. Half of the developers got offices and half got cubes. I had a cube right out in open, offering no privacy. I hated it. I had to learn how to work with a laptop on the lap and I had to be okay with people coming up behind me and commenting on whatever I was looking at. It took a week or so, but I got used to it.

Then the cube next to me opened up. I went in over the weekend and disassembled the wall connecting them. I reworked it so I had a double wide cube with walls toward the door. Privacy achieved. It was great. People could still come by and chat, I had room for a small table to have important 1:1s for discussing XML or Lost. But no one could lurk on me. I felt safe and open.

Then the band broke up a and different people were slung in different directions. I was moved into the same situation I was in when I was hired. I was the only one who had to share an office. I actually made a stink. We were the only people sharing and there were other options. I deserved my own office, right? Probably. But I got stuck with this guy — and this is no reflection on him, he’s great, but still, entitlement. And uh, peace and quiet?

Anyway, this was around the same time I was getting into agile. Really understanding principles of software development. How things really can work well in imperfect environments. Software development, I was discovering, was 90% collaboration/communication and 10% coding. I could go off and continue soloing my projects, or I could discuss what I’m doing more with my team and find better, and often faster ways of doing things.

I came to the realization that the time of the solo coder was gone. An office for a developer no longer makes sense. Developers are the center of work. If they’re inaccessible, they may not be getting the information they need to be doing the work correctly. Which happens constantly. Developers will often run down a rabbit hole head first if it’s an interesting problem. Having an office, with a closed door, is a great way to not be in communication with a team.

The argument most developers make is best explained in the following cartoon to the right:

interruptions

The idea is that we work a lot in our heads. We are putting things together in an organic way. And when someone interrupts that organic process, work is actually LOST.

Interruptions are a part of the job. How we deal with them defines the kind of software engineer we are. To the comic, I would liken it to working on a Word document and not saving. KNOWING full well that at some point someone is going to message you on a chat client that makes Word crash. And autosave isn’t working… okay fine, it’s not a perfect metaphor.

So how do you save?

The concept of saving work can be thought of multiple ways. Rough documentation is always good. Map out your thinking in a document, a google doc or a confluence doc. I create google docs for almost everything nowadays. Some of them end up trash within in day, but it’s good to not lose my train of thought. I can stop anytime, continue it while I’m walking or commuting, pop it out in a meeting that isn’t relevant to me, whatever. You can also talk it over with a coworker. Bringing someone else into your train of thought can not only help you figure out a good solution, but also help in retaining the information. Building a collaborative relationship with your coworkers is paramount to succeeding in software development in the new world. The point is, interruptions are going to happen, so like, save often.

Posted in Uncategorized. Tags: , , , . Comments Off on Offices and Opens and Saving your Work »

Kurogo, Open Source, and Foot in Mouth

Several years ago I worked on a mobile project using modo labs’ kurogo. What we did was simple enough. It provided a mobile framework, using its own phonegap-like technology as a way to “compile” we code into native apps for android and ios. When we were a decade behind in mobility, it provided a means to do things that sounded like they should be done.

Fast forward to present. Modo labs is touting their new v2.0. That’s interesting, right? I knew their 1.x platform and was not impressed, but this is new. In the wake of the new rages in mobility (spa’s and APIs) I couldn’t wait to see what they had come up with.

But wait I did. It took several months to provide me with code to muck with. A project that really enjoys calling itself open source — but they’re not making v2.0 open source. That’s the secret sauce. Okay, fine, maybe they’re having trouble making money with an open source project. I shouldn’t judge it too harshly on that.

Well, I finally got into some training on the product. Got my hands on the code literally 20 minutes before the training. The setup docs were poorly written. In a time where vagrant and docker are on my mind a lot, not having a VM, having to set things up manually seemed like taking a step backwards in time.

Finally got into the nitty gritty, and quickly noticed not much had changed. They are still peddling the same design decisions they made a decade ago. A new enhanced administration, but the apps being created will look pretty much the same. The configuration files write the application for you. As a developer, this was disappointing.

But I realized they’re not making a product for developers. They’re creating a product for configuration managers.

I was floored when I saw PeopleSoft “development” for the first time. It’s the future though. It’s a way to create applications without having to pay for developers with specialized education / experience.

As the people who are putting together the University’s PeopleSoft campus solution I’m sure will tell you, it’s a double edged sword. A LOT can be done by people with minimal training, but getting outside of the box, creating “experiences” becomes borderline impossible.

That said, Kurogo I’m sure has its place among a certain breed of application. And I hope it will take off, and people find use for it. But it is an elegant product, for a more civilized age.

Posted in Uncategorized. Tags: , . Comments Off on Kurogo, Open Source, and Foot in Mouth »

Road to Docker Part 1

Let me be perfectly honest, there will probably not be a part 2. Easy as docker is, there’s still a bit of a barrier to entry. I’m a software dev, I don’t really want to configure vms, but, well, devops. I don’t need to know how to do everything, but I want some understanding of how these pieces fit together. So I took one of our projects and spent some time yesterday to containerized it. I’m going to go through how I went about it.

Step 1. The Search for Tutorials

I spent a while looking through tutorials. Trying to find a good one. Spoiler: I didn’t. I was able to make use of this one mostly: https://www.digitalocean.com/community/tutorials/docker-explained-how-to-containerize-python-web-applications It’s a very long tutorial that I was able to pick and choose some things out of. I’m really sad there isn’t a better tutorial. (That 5 minute docker tutorial is cute, but ultimately worthless.)

Step 2. Installing Docker on a Mac

I wanted to create docker containers on my laptop. It didn’t take long to discover this isn’t really something you can do directly. I wanted to avoid creating a vagrant-virtualbox just to create docker images, so I searched for the standard of what people do. The answer is they use something called boot2docker (https://github.com/boot2docker/osx-installer/releases/tag/v1.4.1), which, as it turns out, just just a linux virtualbox to run docker on. Go figure.

Step 3. Creating a Container — Dockerfile

Upon reading how to create a container manually, it seemed silly to not just create a Dockerfile. A Dockerfile is just a config file that allows docker to create the image. Having mild familiarity with vagrant configs and puppet, the Dockerfile is a way to keep a simple, replicable record of the docker image being created. And simple it is. The format is more readable than it’s cousin, Vagrantfile. It was a simple thing to create the Dockerfile for my Django app. (https://gist.github.com/jazahn/aca49f3e3f9a5b819bce) It uses only a few simple keywords. The most useful being RUN.

Step 4. Creating the Image — build

This was just a simple docker build command

docker build -t docker_image_label .

This creates a new image, labelled with the -t param.

Step 4.5. Finding and Removing Images

docker images

This will list all images (on the boot2docker vm) which you can choose to

docker rm <name>

Step 5. Running the Image — run

This was just a simple docker run command. There are several ways to run, I went with the following:

docker run -d -p 8080:8080 --name docker_instance_name docker_image_label

This just uses -d to run it as a daemon, -p to forward the ports* *It must be noted that “forwarding the ports” is only relevant to the boot2docker virtualbox. To see the result, you should make use of the built in ip reporting for boot2docker.

boot2docker ip

http://viget.com/extend/how-to-use-docker-on-os-x-the-missing-guide

Step 5.5. Stopping Docker, Killing Docker

docker ps

This will give a listing of the running docker instances, so you can

docker kill <name>

Step 6. Docker Hub

One thing that was neato to discover is that knowledge of github / git was translatable to docker. They have designed the usage of Docker Hub around “push” and “pull” concepts. It’s just a matter of pushing images up so they can be pulled down later. What I wasn’t able to find was how to actually send the Dockerfile along with the image. It’s not strictly necessary, but I’ve seen it on other projects, so I know it’s possible…

Conclusion

Getting the container up and running and distributable was no big deal, but this was not enough. Next steps would be to grab a mysql docker image from the hub and have the two communicate with each other. Right now I’m really just using the container as a VM, which isn’t really the point of docker.

Posted in Uncategorized. Tags: , , , . Comments Off on Road to Docker Part 1 »

The Silo Mentality: It Can Be A Good Thing If You Don’t Do Your Own Thing

Silos: What Are They Good For?The Silo Mentality,  as defined in the Business Dictionary, is not new nor confined to business. Silos are prevalent in all aspects of Academia, and having them around is generally considered a Bad Thing. Take a certain University in Cambridge, for example. It’s a big place. A huge place. With many different Schools, consisting of many different administrative and academic departments, each consisting of many different groups. Not to mention libraries and museums and other centers of learning. It’s a cornucopia of conflicting visions, motivations, and group mentalities, all of which are somehow supposed to work together toward a common mission of education and research (which, in themselves, are at sometimes at odds with each other). Having each entity operating by itself, looking inward rather than outward, seems a recipe for disaster.

A disaster it can certainly can be, and there is often great expenditures in time, money, and people to break down silo barriers. For those of us in Educational Technology, the proposed wrecking balls are often getting everyone onto the same Learning Management System (LMS), the same calendar, the same email, the same network, the same software and hardware  and computers and . . . the list goes on and on. For administrative functions, such as calendaring and email, this approach makes sense. It’s incredibly difficult to operate  with different calendaring and email systems. Even using different version of office software or operating systems can make a mess of things (I’m looking at you Microsoft Office for Mac and Windows!).

So I can understand why academic organizations such as University IT may want to centralize, well, everything they possibly can, including environments for software development. I’ve heard many of my co-workers and leadership say things like, “We should all be using Java” or “We should all be using Git” or “We all need to be using Agile Software Development.” At the surface level, such decisions make sense. If everyone uses the same tools and possesses a similar skill set, then you have an excellent pool of resources that can look out for each other. If one person gets sick or gets hit by that proverbial bus, someone else can jump in. You can shift people on and off projects more efficiently. Getting up to speed is faster. Yes, there’s lots of good stuff there.

Such a centralized approach may be good for a small business or even a small school, administratively. But in Higher Education, even at small schools, the breaking of barriers argument quickly implodes when it comes to academic needs. This is because Higher Education is, by definition, about academic freedom, and it’s this freedom, as ironic as it may seem, that builds the silos in the first place. The building of silos, in fact, is inevitable and necessary.

As an example, take software development for pedagogical tools, an area my group is intimately familiar with. Unlike many of the software development groups within our parent organization, we do not focus on the development of enterprise-wide applications that cater to a common need across multiple organizations; we develop software on a much smaller scale over a much more diverse range of needs and, hence, technologies. Additionally, we are responsible for helping faculty and students stand up their own software, which usually involves a different technology stack than our own. So unlike some of our counterparts, we cannot be confined to a single language such as Java or Python. We cannot be confined to single deployment infrastructure. Such confinement inhibits the academic freedom of students and staff. We are, by necessity, different from other groups. Separated. A silo.

This is not a Bad Thing. Our educational technology group needs to operate differently from enterprise groups–such as, say, the Registrar–just as the enterprise groups need to operate differently from us. Maintaining these silos is a Good Thing. It’s isolating the silos from each other that causes problems.  Since our group is building course ware, we are often interested in accessing student data. It would make zero sense for us to have our own set of student data when we should be utilizing the information managed by the Registrar. If we did our Own Thing, that would be isolation.

It’s isolation of silos that causes people to start waving around those wrecking balls of centralization, but removing the silos all together is missing the point of academic freedom. Rather, bridges need to be built between the silos, conduits of communication and best practices that allow the silos to act as a cohesive team. So how does one go about doing this? We’ll explore some possibilities in  an upcoming post. Maybe. Assuming I write it.

Posted in Uncategorized. Comments Off on The Silo Mentality: It Can Be A Good Thing If You Don’t Do Your Own Thing »

Topics in Version Control

versioncontrol-allthethingsThis last month I put together a presentation on version control. As far as I’m aware, everyone I work with uses some form of version control. My purpose with the presentation was many fold. I and a lot of thins that had been prrcelating that I wanted to think about and express. This as an excellent opportunity for that.

Git vs SVN

One thing I know is a sore point for some developers is the idea that git is taking over. There are a lot of people espousing the idea that git is better than svn in every way. This has created a conflict culture because there are a lot of developers making good use of svn and most have heard the primary arguments in favor of git and found the arguments are not compelling enough to motivate a switch.

This is okay.

There is no war on centralized version control, and if there is, the only people pushing it are dumbasses. (This coming from someone who was one such dumbass a couple years ago.)

It’s anti-agile to prescribe something like that. Each team works differently and needs to determine for themselves if the benefits outweigh the costs.

Continuous Delivery

Recently I read a book called continuous delivery. It is a comprehensive blowhard description of how devops “needs” to be. It covers testing strategies, and branching strategies. While a lopsided view of the world, it offered potential solutions that I have found fascinating.

Trunk Based Development vs Feature Branching

This offers not just a workflow that allows for potentially faster delivery, but also a realistic way to use SVN. An alternative to the branching hoopla around distributed systems.

DevOps

It also gave me an opportunity to talk about DevOps in a reasonably safe environment. It’s a political word at the moment and while I don’t want to get involved in that, I do want to talk about the DevOps movement.

This movement is about working together. Communicating on a level that hasn’t classically been done. It’s there because a lot of the problems we face are due to the fact that in our workflow, we develop a product and throw it over a virtual wall to operations to “handle”. They don’t understand what we need done and we don’t understand what they need done. It’s a matter of sharing knowledge and searching for a deeper understanding of what we’re doing. It’s hard because we don’t want to spread ourselves thin, but in order to deliver a quality product, you have to be more than just a cog in the machine. You don’t have to know exactly “how” to do everything, but you have to have a basic understanding of “what” is happening.

But I digress, I was able to tie this in to version control by talking about how DevOps is also concerned with automation. Creating scripts to do all parts of the deployments. These scripts, this automation needs to be in version control as well. They “can” be in the same repository, or they can be in a separate repository. But managing dependencies, flipping all the right switches, needs to be maintained in much the same way code is controlled. Because it is code.

Database Version Control

And of course, you can’t talk about Version Control without talking about database version control… but this one requires its own article.

Posted in Uncategorized. Tags: , , , . Comments Off on Topics in Version Control »

Cloud Delivery Platform — Thought Model

I was asked to take a look at a thought model for a Cloud Delivery Platform. I’m not going to post that graphic, because it wasn’t ready for prime time. It was confused and I’m not really sure what it was trying to show. So I created a bit of a diagram showing what I thought was important. (Just click on it, it doesn’t seem to fit right, and the next size down is too small to read.)

cloud_delivery_platform

Halfway through I realized I’m not terribly good at making these things. I’ll get better. But there’s a few things I wanted to draw attention to in this graphic.

The Roles

This was the most important thing to me is to highlight the crossover of roles for each step. There are few steps in the process that should be perfomed by a single role. Devops is about communication.

Local Development

Coding, building, testing, packaging are all things that should be done at the same time. You don’t code 1000 lines then build. You build with every small change. In the same way, you should have tests written with the small increments. And that gets extended to “packaging”.

Packaging

I’m probably missing a very important word in my vocabulary here, but I’m going to go easy on myself, it’s late. What I’m meaning here is the process of setting up an image, or a deployable product. The configuration that will collect external dependencies and run tests when it gets to the deployed environment. This should all be done locally first. It should all be put into version control, separate from the application code. And this needs to be done in collaboration between the developer, who knows the application, and operations (or as they’ve been renamed in my organization, “Devops Engineers”) those who know how to package things appropriately.

Local Concurrence vs Cloud Linearity

Why isn’t spell check underlining “linearity”. That can’t be real. As I mentioned before, the things done locally are all done at the same time (build, test, package). That’s all just development. When it gets to the cloud, it’s a done product. Nothing should be further developed there. Nothing should be getting reworked. Everything needs to come entirely from Version Control and no manual finagling. (obviously?) So the cloud portion is linear. Everything happens in order. If it fails at any point, it’s sent back to local dev and fixed there.
Maybe this is all so obvious, it doesn’t need to be said. Maybe I’d feel better if it were just said anyway.

Posted in Continuous Integration, Uncategorized, Version Control. Tags: , , , , , , , . Comments Off on Cloud Delivery Platform — Thought Model »

Jira vs Github :: Agile vs Open Source

A few years ago, my team got into Open Source. Specifically, we started writing all of our apps on github (as opposed to our SVN). We wanted to do this because we wanted to invite scrutiny. We never expected people to look at our stuff, we just felt that by putting it out in the open, we’d want to do better internally.

We went whole-hog. Organized stories with issues, organized sprints with milestones. It was pretty hot stuff. And it was all in the open. Potentially, someone could come by, see what we’re doing and offer to take a story from the backlog of issues. That’s open source. ish. We have a lot more we can do. A lot of growth to do.

Then we started using Jira. The board system within Jira Agile was excellent. Allowed for better organization, reporting!, and visual representations of work. It’s great. It’s Agile. But it also replaced what we were doing with Github issues.

We essentially replaced Open Source in favor of Agile. Our organization is great, we’re keeping track of things fantastically, but we’re no longer open. We don’t have transparency on what we’re working on anymore. People can’t potentially help. Our code is out there, but we’re not inviting. Our process is no longer out there.

So what’s our solution? We don’t have one yet. But what /can/ we do?

We need to put our vision statement out there. We need to put our plans out there. We need to expose what it is we’re doing. We also need to stay agile, keep our tools intact, keep our reporting.

This means we probably need to be duplicating efforts. Open Source and Agile are both hard work and organization. That they can’t line up and be the same effort is not a blocker, just an “oh well”.

Posted in Agile, ATG, Git, Open Source, SVN. Tags: , , , . Comments Off on Jira vs Github :: Agile vs Open Source »

Enterprise

enterprise_1920x1200It’s goal setting season and everyone is reviewing the organization’s FY15 goals. Understanding the organizational goals is important because we all want to be sure to be in line with the direction of the organization.

#4 on the top 10 is “Establish an enterprise IT architecture”. This is a phrase that I’ve never fully understood. So I’ve been asking people to define “enterprise”, I get all sorts of answers. “big” “important” “java / oracle / peoplesoft” “something that ‘can’t’ fail” “finance” “HR” “business”. (Important was my favorite.)

People use this word constantly. There’s even groups in my organization that are called the “enterprise groups”. I was told recently that there’s “less dysfunction in the enterprise groups”. To which I had to ask “which groups are those?” (Answer: They’re the ones that have significant business value.) But the fact that I can’t find someone to give me a coherent definition is a warning sign that it has become a meaningless buzz word.

But business is the right answer. Enterprise seems to be anything that encompases the entire organization (or a significant portion of it) or something that has significant business value.

Anyway, that’s neither here nor there. What I think someone was trying to say with “enterprise IT architecture” is a standardization of technology across the organization — often a pipe dream. And I think this goal tends to draw extra attention to the groups that are already considered the “enterprise” groups.

But I’m begging a deeper question here. Shouldn’t everyone be considered enterprise? If enterprise is just a fancy word for business. If you’re not providing significant business value to your organization, then maybe you need to rethink what it is you’re doing. If you’re not considered enterprise, then someone doesn’t think you’re providing business value, effecting enough people or having enough impact.

So when “establishing an enterprise IT architecture”, I’m taking that to mean communicate with other groups, figure out what they’re doing and share what you’re doing. Meeting that goal is potentially impossible with a large silo’d organization. But working toward that goal is ambitious and requires a lot of time. Working towards that goal is just implementing good collaborative practices. Simple but not easy.

Posted in Uncategorized. Tags: , , , . Comments Off on Enterprise »