Engineering is all about making informed choices in the problem domain. Software Engineering is about choosing speed vs. space, simple vs. complex, flexible vs. fixed, etc. based upon the project requirements, specifications and the training and experience of the engineer. This page lists many aspects of Software Engineering but unfortunately many coders only know a few of these and that limitation affects the quality of their work. I call myself a software engineer and architect because I apply all of my extensive training and experience to my projects.

I use the term "granularity" to express different levels of looking at a software solution. In a computer granularity can range from microcode (in CISC chips) to assembler to C to Perl to modules to complete systems. I view software architecture, design and coding as three different levels of granularity of the same problem and similarly for system, integration and unit testing. Choosing the best Granularity to attack a software issue is paramount if you want to create a effective and maintainable system.

Architecture is the highest level of granularity in software engineering. Architecture defines how the system is broken into large scale components and how they interact with each other. The result of an Architecture are the public APIs and interfaces, modules and classes, communication pathways, data flows, etc. Too often developers skip the Architecture phase and delve directly into design and coding but they pay for it later on. A good Architecture makes for a cleaner design and much lower maintenance costs. All of my development projects have been architected first based on proper specifications and that has been proven to be beneficial over time. I put a larger portion of the project schedule into architecture than many others do, as much as 20%. When I created Stem, I spent three months (part time) working out its Architecture before I even wrote anything down. The design and coding of the first version took only three more months which reflects on the effort put into the architecture.

I use the term "Universal API" for the architectural motif where each component can interact with any of the others in a standardized way. The most well known universal API is in the set of common text stream utilities on *NIX systems such as head, grep and sed. They all can be used in pipelines and read their STDIN, process text and output to STDOUT. Each such filter used in a pipeline doesn't know anything about the neighboring programs - that is only known by the higher level shell script that creates the pipeline. Another example is Stem, where its modules all communicate via message passing (two-way vs. one-way for shell pipelines) and each module doesn't know nor care to whom it is communicating. And Stem's configuration files which specify the modules and their connections to each other are analogous to the shell pipeline commands. A universal API is a great architecture when you need to create a very flexible and powerful system but want to keep the components small and focused. See extreme modularity for another useful architectural idea.

I use the term "extreme modularity" to describe a way of creating small focused components in an architecture. It means the components should be as smart as possible about a very small problem space. As with a universal API, *NIX utilities and Stem are good examples of this concept. *NIX utilities such as head knows how to get the first lines of a file and not much else and it does it well. Stem's Stem::Proc module knows how to fork a process, manage its I/O and other parts of it. It doesn't know nor care which process it runs nor how it interacts with other Stem modules as that is handled in the higher level configuration. The key to a good extremely modular architecture is choosing the right granularity of the components. Too small and you need to use and interconnect too many components. Too large and you defeat the purpose of focused and small components by creating monolithic modules.

Design is a middle level of granularity in software engineering. It is the translation of the architecture in more concrete results such as algorithms, data structures, non-public APIs and interfaces, and other lower level components. This is where you concern yourself with efficiency and resourse usage. An important aspect of design is that it should be language independent. Many developers design in their target language and that can affect things for the worse because they let a coding implementation change the general way something should have been done. You shouldn't do (much) coding when you are desiging a system. But actual coding from a design should be very straightforward and not be a complex task. Also testing is usually designed in parallel with the design of the main project.

Coding is the lowest level of granularity in software engineering. It is where you translate the design into actual code of the chosen language (which should be Perl if possible! :).

Reviewing and commenting on Perl code is one of the most enjoyable support services I do. That is why it it a key Perl On Call service and has its own page.

Refactoring is the process of iteravely rewriting reorganizing code while keeping its current API stable and also consistantly passing a set of regression tests. This is another service that can be done incrementally and is an extremely valuable way to improve your code base and skills. An excellent book on refactoring and dealing with legacy Perl systems is Peter Scott's Perl Medic.

Best practices are a collection of coding rules and styles adopted to make your coding more consistant. They can be a short list used by one developer or a long manual used corporate wide. There isn't one fast set of Best Practices as they need to be created to best suit the development team's needs. A very good resource for this is Damian Conway's new book called (what else? :) "Perl Best Practices" which is due out in August 2005. (Read the FAQ for a surprise regarding this book!).

A system is a set of components and their relationships. Software components are typically processes, modules and objects. Their relationships are communication, APIs and interfaces. Creating a software system is the primary goal of software engineering.

This is a subject too big for a short paragraph and so it has its own page.

Regular expressions are one of the key components to Perl and can be difficult to master. I can help solve your regular expressions problems, improve their efficiency, explain complex ones, and teach you ways to write better regular expressions.

Technical documentation is hard to make readable and also be accurate. I have been a technical editor of four Perl books and have written several articles and talks on Perl. My English skills mesh with my technical understanding to allow me to create and edit coherent and clear technical writing.

Stem is a network application toolkit and framework I created. It emphasizes the two architectural concepts of extreme modularity and universal API. It uses a high level configuration to specify module instances (objects) and how they interconnect with each other. Inter-object communication is via message passing which is its universal API. Stem's modules are architected to be very smart about focused issues which is the goal of extreme modularity. Stem can be used to create simple or complex applications with much less work designing and coding due to its powerful architecture. Stem can be found on CPAN and you can read more about it at stemsystems.com.

Perl comes with over 1000 pages of free well written and accurate documentation. But that can still be daunting to Perl users who are learning the language. I can help point out the manual pages that are related to the topics you are learning and also help clarify anything that is not completely clear to you.

There are over one million Perl programmers world wide and they form a very active community. On-going activities include maintaining Perl 5, developing Perl 6/Parrot, CPAN, conferences, local user groups (Perl mongers) dozens of mailing lists, web sites, books, articles, etc. I am very active in the Perl Community in many areas and I can help your shop become more involved with it. The Perl Community is a very valuable resource and is something that other language groups try to emulate (but not always successfully :-).

I am not a classic 9-5 classroom trainer. I teach by using a wide range of resources such as email, IRC, talks, articles, books, and documentation. I work with your staff on helping them learn what they need to solve their Perl problems. I call this Integrated Training since it combined those resources with techniques such as code review and by using your own code base as the source for examples. Perl on Call is a very cost effective way to provide your staff with Integrated Training.

No one learns all they need from a single class. Learning is incremental process and Perl On Call is a great way to support it. On a day to day basis, your staff is able to ask questions and getting good answers. Perl topics important to your staff will be brought up and discussed so that everyone can learn new useful techniques. Incremental Learning naturally occurs in the on-going relationship that I will build with your staff.

CPAN is public repository of free to use Perl modules and is a resource that is the pride of the worldwide perl community (and the envy of other languages). But can be hard to navigate and to quickly find the best module for your needs. In some cases, there are multiple modules which overlap in function. I can help you find the module for your needs and help you learn how to best use it.

For some reason Perl has an ill gotten reputation for being tough to hire programmers. I helped alleviate this when I created (and still co-manage) the Perl jobs list (jobs.perl.org). It currently averages over 80 job postings a month and has thousands of subscribers world wide. Beyond that, accurately assessing the Perl (and general programming) skills of applicants is difficult. I can help with resume and candidate reviews and other aspects of the hiring process.

Requirements are what the your client or boss tells you what they want. They are should be written from the point of view of the end user and/or the goals of the product/project. Too often requirements are very vague and poorly written. It becomes a major task for the developer to analyze them and convert them into proper specifications

Quality Specifications are a must for any properly engineered system. They are used both to guide the architecture and design as well as the test plans. Loose specifications leave too many open issues that make the development tricky and fraught with testing issues. Too much detail in a specification can constrain architects and designers and limit their engineering choices. As with other software engineering decisions, the right level of granularity in a specification is important.

The FAQ-O-Matic is a simple tool I created that will email Perl FAQ entries to your staff on a daily basis. This has proven to be an excellent training method and it instigates many interesting threads.