Tech bites

• Jan 27, 2026
• python , machine learning , optimization

Optuna - Hyperparameter Optimization

Optuna is a Python library for automated parameter optimization using adaptive search instead of manual tuning.

It shines whenever:

• You have multiple knobs to tune.
• The system returns one final numeric score.

This pattern shows up everywhere: machine learning metrics, simulation outcomes, and trading strategy ROI.

What Is a Hyperparameter?

A hyperparameter is a setting you choose before running an evaluation.

Examples:

• A model’s learning rate or tree depth.
• A trading rule’s lookback window or threshold.
• A simulator’s step size or penalty weight.

Unlike learned parameters (like model weights), hyperparameters are not fit directly by gradient descent. You set them, run the system, and observe a final score.

• Nov 30, 2025
• Math

Calculus Refresher: The Ultimate Formula Cheat Sheet

A reference of core formulas and concepts that build on each other — perfect for brushing up before advanced math, physics, or machine learning.

1️⃣ Limits & Continuity

Limit definition
lim_{x -> a} f(x) = L

Common limit laws

• Sum: lim(f + g) = lim f + lim g
• Product: lim(f * g) = (lim f) * (lim g)
• Quotient: lim(f / g) = (lim f) / (lim g) (denominator ≠ 0)

Special limits

SQLAlchemy vs Hibernate: A Deep Dive Into Python and Java ORMs

Introduction

When developing applications that interact with databases, Object-Relational Mappers (ORMs) help bridge the gap between relational databases and object-oriented programming. Two of the most popular ORMs are SQLAlchemy (for Python) and Hibernate (for Java). While both serve the same purpose, they have different approaches, strengths, and best use cases.

In this post, we’ll compare SQLAlchemy and Hibernate, provide code examples, and show how to generate SQLAlchemy classes from an existing PostgreSQL database using sqlacodegen.

• Feb 28, 2025
• AWS , Cloud Storage , Databases

Comparing Persistent Data Sources on AWS: Choosing the Right Storage Solution

Introduction

Amazon Web Services (AWS) offers a variety of persistent storage solutions designed to meet different needs, from simple object storage to high-performance databases. Choosing the right data source depends on factors such as scalability, durability, cost, and access patterns. This guide compares the most commonly used persistent data storage options on AWS to help you make an informed decision.

AWS Persistent Storage Solutions Overview

Here’s a high-level comparison of AWS’s persistent storage options:

• Feb 12, 2025
• ai

Write Unit Tests First and Use AI to Generate Code That Passes Them

Introduction

Many developers turn to AI to generate unit tests for existing code, but what if we flipped the process? Instead of using AI to write tests, we can write unit tests first and use AI to generate the implementation that satisfies them. This approach aligns with test-driven development (TDD), ensuring that our code meets predefined requirements and is robust from the start.

In this article, we’ll explore how to use Java with Gradle and JUnit to write unit tests first, then leverage AI to generate code that fulfills the tests.

• Aug 15, 2024
• python

Walrus Operator

The walrus operator (:=) in Python, introduced in Python 3.8, is also known as the assignment expression. It returns the value as well as assigning it. This can reduce many lines of code and greatly simplifies the language.

Example

# Without Walrus
numbers = [1,2,3]

numbers_length = len(numbers)
numbers_sum = sum(numbers)

numbers_description = {
    "length": numbers_length
    "sum": numbers_sum
}

# With Walrus - see how we save the variables and return the value
numbers = [1,2,3]

numbers_description = {
    "length": numbers_length := len(numbers)
    "sum": numbers_sum := sum(numbers)
}

Another Example

# Example code to read lines from a file and process non-empty lines
with open('example.txt', 'r') as file:
    while (line := file.readline().strip()):
        print(f"Processing line: {line}")

• Jul 30, 2024
• hatch , python

Managing Hatch Dependencies in vs Code

I recently started using hatch for python projects, and even though hatch will manage and install dependencies for you (just modify the pyproject.toml file), I noticed the import statements could not find the dependencies. This has to do with how Python and Hatch create virtual environments.

To fix this:

CMD + Shift + P
Python: Select Interpreter
Find the Python Environment that matches the Hatch environment.

• Jul 25, 2024
• git

Git Hooks

Git hooks are scripts that Git automatically executes before or after specific events, such as committing changes or pushing to a repository. They allow you to customize and automate tasks related to these events, such as enforcing code style rules, running tests, or sending notifications. Hooks are useful for maintaining code quality and ensuring consistent workflows across a team. They can be set up in the .git/hooks directory of your repository and include both client-side and server-side hooks.

• Jul 15, 2024
• hatch , python

Hatch - A Great Python Project Management Tool

Hatch is a python packaging tool. Useful for building python projects. Out of the box it supports testing, building, managing dependencies, and linting/formatting support.

https://wwww.hatch.pypa.io/latest.intro

New Project

hatch new “Hatch Demo”

Adding to Existing Project

hatch new –init

Build a project

hatch build

Test a project with coverage

hatch test –cover

Running static analysis

hatch fmt

Open a shell in the project to run scripts

hatch shell

Create a python virtual env

python3 -m venv /tmp/hatch_demo/

• Jun 25, 2024
• Programming

Recursion

Definition of Recursion:

Recursion is a programming technique where a function calls itself in order to solve smaller instances of the same problem until it reaches a base case that does not require further recursion.

Example in Python:

def factorial(n):
    """Calculate the factorial of a number using recursion."""
    if n == 0:
        return 1
    else:
        return n * factorial(n - 1)

# Example usage:
result = factorial(5)
print(f"The factorial of 5 is: {result}")

In this example, the factorial function calculates the factorial of a number n recursively. It calls itself with n - 1 until n reaches 0 (base case).

• Jun 17, 2024
• Command Line

How to Search (And Replace) Quickly Across Directories

To find things quickly - use the silver searcher.

brew install the_silver_searcher

To find things:

ag 'the thing to find'

To see the count:

ag 'the thing to find' --count

To see the list of files only and search on a literal string:

ag 'the thing to find' -Q -l

To then replace the occurrences of the found use:

ag 'thething' -Q -l|xargs sed -i 's/thething/bar/g'

• May 01, 2024
• Java

Safely Retrieve Properties Using Supplier and Optional in Java

The supplier allows you to pass something but not retrieve it right away. One use case is retrieving a property an object that could throw a null pointer exception. By not retrieving right away, but instead using a supplier, the supplier can be later executed inside a try/catch block and handle any exceptions thrown.

public void example() {
    // Without supplier - could throw a nullpointer exception on any getter
    try {
        String str = someObject.getLevel1().getLevel2().getLevel3().getString();
        someOtherObject.appendValue(str);
    catch (Exception e) {
        // Exception throw - do nothing
        // This code is ugly!!!
    }

    // With supplier
    Optional<String> result = resolve() => someObject.getLevel1().getLevel2().getLevel3().getString());
    // Now we can do something if the property is present or skip if it doesn't exist
    // Much cleaner code
    result.ifPresent(someOtherObject::appendValue);
}


// This resolve method can be reused
protected <T> Optional<T> resolve(Supplier<T> supplier) {
    try {
        T result = resolver.get();
        return Optional.ofNullable(result);
    } catch (NullPointerException) {
        return Optional.empty();
    }
}

• May 01, 2024
• Java

Function Composition in Java

An incredible tool in Java is the ability to create functions and compose them.

One common use case is to extract a variable from an object and then transform it.

In the object oriented paradigm, you would have a method that extracts and another that transforms. Then intermediate variables would be used to convert from the initial object to the final transformed result.

With function composition this is greatly simplified. The separate steps can be defined as functions and then composed together to create the chain of events that need to occur. Then, the final composed function can be used instead of manually calling each method. See the examples below.

• Feb 21, 2024
• Tooling

Eclipse Gotcha: Refactor->Move Not Showing Up in Git

In eclipse, if you refactor->move a file from one project to another, it may not show up in your eclipse git staging as deleted from the original project.

To Avoid: Copy the file to a different project and then delete from the original.

To Fix: Use the command line to delete the file (it will no longer show up in eclipse, but the command line will display it) and then use git from then command line to push the change.

• Feb 09, 2024
• Javascript

Set Interval Timer

Be careful when using setInterval as a timer.

let timeInSeconds = 0;
let interval = setInterval(() => {
    someFunction();
    timeInSeconds++;
}, 1000);

While it appears your time increments every second, it will actually increment one second (1000 milliseconds) plus the time the someFuntion() takes to run. This timer will eventually be out of sync.

A better approach is to create a point in time when the timer starts and then subtract the difference anytime the timeInSeconds is updated. In this scenerio, you can set the interval to any milliseconds amount and get an accurate time everytime the timeInSeconds variable is updated.

• Jan 11, 2024
• Front End Development

React Reducer Runs Twice

If using the react hook useReducer(reducer, state) and you notice it runs twice after dispatching an action - this is actually by design for development mode.

The reducer dispatches an action with a given state and returns a new state. It should never mutate the previous state.

If you follow this rule, you’ll never notice the reducer running twice. However if you are mutating the state and returning the new state based off that, you’ll make the mutation twice. So if you are adding something to a list in the state, you’ll see it added twice. This feature of react development keeps our functions pure. Pure functions are much easier to test.

• Dec 29, 2023
• JPA

Maintaining List Order in Jpa

If you want to maintain the order of a list of objects retrieve from the database using hibernate/jpa, use the @OrderColumn annotation. An additional column will be created in the database (if ddl is set to update) that will keep track of the order/position of an item in a list.

CAVEAT: If you change the position of an item in a list OR delete an item besides the last one, it will cause a multi update across the list to update all other positions. Be aware of this intensive operation for large lists.

• Dec 28, 2023
• Database

Postgres: Useful PSQL Commands

Connect to a database

psql -d dbname -U password

Create a database

CREATE DATABASE mydb;

Create a user

create user myuser with encrypted password 'mypasswd';

Grant Privileges

grant all privileges on database mydb to myuser;

To list databases

\l

To list users

\du

List all tables

\dt

Describe tables

\d

List schemas

\dn

Switch database

\c dbName

Run psql commands from file \i fileName

Check version

SELECT VERSION();

Quit

\q

• Dec 19, 2023
• Security

SQL Injection

SQL injection occurs when SQL code can be injected into API input. In this injection attack, valid input has SQL commands concatened with SQL execution commands. When the SQL code is executed, the commands are run. In this process data can be mutated and returned to the attacker.

Mitigation Tips:

• Use prepared statements instead of concatenated SQL statements. This seperates inputs from the command.
• Restrict the user account that the SQL command is executed with to only allowable actions.
• Always sanitize input on the server.
• Never trust the client input to have sanitized input, it can be exploited by an attacker.
• Use mature libraries for data sanitization, there are too many variants to look for to write custom sanitation code.

• Dec 18, 2023
• Security

API Security

Always remember the CIA triad when securing API’s:

• C: Confidentiality - Only intended audience can access information.
• I: Integrity - Prevent unathorized mutation of data.
• A: Availability - API can be reached by legitimate users.

Common API Threats: STRIDE

• S: Spoofing - pretending to be someone else
• T: Tampering - altering data
• R: Repudiation - denying authorship
• I: Information disclosure - revelaing private information
• D: Denial of Service - preventing access
• E: Elevation of Privilige - gaining access to unauthorized information

• Oct 22, 2023
• Front End Development

React Hook Example: UseState

Below is an example of React hook useState to change the state of a boolean.

const MyCustomButton: React.FC<Props> = props => {
    // first argument defines the variable
    // second argument defines the funtion that changes the state
    // third - the assignment sets initial state
    const [open, setOpen] = React.useState<boolean>(false);

    return (
        <Button
        onClick={() => setOpen(true)}  // calls the function that will set the state
        >
    );
};

• Oct 12, 2023
• Back End Development

Auditing With Envers

You can audit any changes to any entity using envers.

Dependency and configuration found at: https://hibernate.org/orm/envers/

Then annnotate entities to audit:

@Entity
@Audited
public class Employee { ... }

Now when you make changes to an entity an audit entry will be created automatically.

Employee emp = employeeRepo.findById(1L);
emp.setSalary(75000);
employeeRepo.save(emp); // audit table will be updated here automatically to an audit table

Audit tables can be created automatically by Spring/Hibernate with hibernate.hbm2ddl.auto=update

Envers is a useful library for building in auditing into your existing Hibernate application.

• Oct 11, 2023
• Programming

Becoming a Professional Software Developer

“Everybody wants to be a bodybuilder, but nobody wants to lift heavy ass weight”

Ronnie Coleman, 8x Mr. Olympia winner

Being a programmer is a lot like being a bodybuilder. Regular training, incremental progress, over and over again.

Over time this transforms the person from a beginner to a profesional.

Adopt this mindset in being a programmer. Plan to consistently train and improve your software engineering skills.

• Oct 10, 2023
• Front End Development

Axios Promise - Request and Response From the Client

Simple example on how to send a post request from the client and handle the response.

Example

axios.post('/login', { firstName: 'Jack', lastName: 'Johnston' })
.then((response) => {
    // Success
  console.log(response);

}, (error) => {
    // Failure
  console.log(error);
});

Further Documentation

// Send a POST request
axios({
  method: 'post',
  url: '/user/new',
  data: {
    firstName: 'Joe',
    lastName: 'Smith'
  }
});

Install instructions

# NPM
npm install axios
-- or --
# Include Script Tag in JS file
<script src="https://cdn.jsdelivr.net/npm/axios/dist/axios.min.js"></script>

• Jun 22, 2023
• Git

Git: Add Date to Git Commit

To add a date to a git commit message from the command line:

git commit -m "blog update on `date +'%Y-%m-%d'`"

will result in

* e81bc5d - (25 minutes ago) blog update on 2023-04-20 - Mark S

• Jun 08, 2023
• Kubernetes

Increase Pod Memory on the Fly in Rancher

1. Select Deployments from menu-sidebar
2. Find correct deployment and click on three dots
3. Edit Config
4. Select Resources from deployment-sidebar
5. Modify memory reservation and limit here
6. Save

• Jun 06, 2023
• Spring

Spring: Enable Components and Job Schedulers With Spring Profiles

In Spring you can enable a component using a profile:

@Profile("prod")
@Component
public class ProductionService() {
    // This will only be created with Spring_Profile: prod enable
}

Another use case is enabling/disabling schedulers:

@Profile("nightly-job")
@Component
public class NighlyJobScheduler() {
    // This will only be created with Spring_Profile: nightly-job
}

It can get cumbersome adding all the various profiles when you run the application. Spring now offer profile groups:

spring:
  profiles:
      active: development
      group:
        prod: prodJob1, prodJob2
        development: dev-scheduler-1, dev-notification

In the example above, running the profile “development” will automatically bring in the other profiles “dev-scheduler-1, dev-notification”

• Jun 06, 2023
• Programming

YAGNI

YAGNI is an acronym for “You Aren’t Gonna Need It”.

It’s an important principle in software development. Often it’s tempting to add code that could have useful utility down the road. In the end, it never ends up being used and creates overhead and code bloat.

So to follow YAGNI, don’t implement code until it actually needs to be utilized.

It’s a principle that comes from XP - Extreme Programming.

• Jun 06, 2023
• Kubernetes

Kubernetes: Restarting Pods

When making changes to config maps or other things - it’s tempting to restart all the pods for a particular service (select them all, delete, and let them come back up). This may work in the majority of cases, but if something goes wrong, it’ll be hard to tell if the recent change caused it or if it’s unrelated.

Recommendation:

• First check the logs in the pods to restart for any abnormal behavior
• Second restart a single pod before making any changes, make sure it comes up properly
• If the pod cannot restart without making any changes, debug the issue
• If the pod can restart, make the config change.
• Now restart (or kill) pods one at a time, ensuring each one enforces the new change.

The point here isn’t the specific process, but rather having checkpoints when changes are made. Ideally, by making only one change at a time and testing, it’s evident when something is broken which change caused it.

• Jun 05, 2023
• Kubernetes

Access Command Line in Kubernetes Rancher

To access a shell in a particular pod:

• Workload-> Pods -> Find pod -> Click Three dots -> Execute Shell

To access a terminal to run kubectl commands in the environment:

• Click the >_ symbol to open a terminal.

Example command: kubectl describe thingToDescribe -n NameSpace

• May 30, 2023
• Batch Processing

Spring Batch: Running a Batch Job Explained

To run a batch job, a JobLauncher needs a batch JobConfiguration and JobParameters. This creates a JobInstance.

A JobInstance creates a JobExecution. If the job fails and is restarted it retains the same JobInstance but creates a new JobExecution.

A JobExecution runs one or more steps in the Batch_Step_Execution table.

So each new run of a batch job creates a new batch job instance, and each restart creates a new job execution.

• May 19, 2023
• Git

Git: Checkout a Single File From Another Branch

If you need to checkout a single file from another branch in git use:

git checkout BRANCHNAME --path/to/file

• May 18, 2023
• Batch Processing

Spring Batch: Creating a Custom Job Builder

Batch jobs are created in Spring Batch using the JobBuilderFactory. A custom job builder factory can be created and re-used to allow common job configurations to be shared across all batch jobs in a project.

Example

@Configuration
public class CustomJobBuilderFactory extends JobBuilderFactory {

    @Override
    public JobBuilder get(final String name) {
        final JobBuilder jobBuilder = super.get(name);
	// register common to all jobs things here
        jobBuilder.listener(commonListener);
	return jobBuilder;
    }
}

Implementation

    @Bean
    public Job exampleJob(final CustomJobBuilderFactory jbf) {
	// This job now has the common listener 
	return jbf.get("myJob").start(myStep).build();
    }
}

• May 18, 2023
• Batch Processing

Spring Batch: Trigger an Action When a Batch Job Fails

When a batch job fails it’s useful to automatically send a notification or trigger some sort of an action.

Spring Batch allows registering a listener in the job configuration. Below is an example of a listener that triggers an action after a job. Using the job execution, the action can be set to only fire when the job is not complete.

@Component
public class AfterJobListener implements JobExecutionListener {

    @Autowired
    private NotificationService notificationService;

    @Override
    public void afterJob(final JobExecution jobExecution) {
        if (jobExecution.getStatus() != BatchStatus.COMPLETED) {
	    notificationService.sendNotification(); // pass data about the job here
	}
    }
}

• May 08, 2023
• Java

Convert String Into LocalDate

Code

   public static LocalDate parseDate(String date) {
        DateTimeFormatter formatter =
                 DateTimeFormatter.ofPattern("M/d/y");
        return LocalDate.parse(date,formatter);
    }

Unit Tests

    @Test
    public void testDateParsing() {
        String testDate;
        testDate ="01/01/2022";
        LocalDate expected = LocalDate.of(2022, 1, 1);
        assertEquals(expected, DateParser.parseDate(testDate));
    }

    @Test
    public void testDateParsingSingleDigitMonthAndYear() {
        String testDate;
        testDate = "1/1/2022";
        LocalDate expected = LocalDate.of(2022, 1, 1);
        assertEquals(expected, DateParser.parseDate(testDate));
    }

• May 03, 2023
• Programming

A "Simple" Change to a Web App

Imagine a basic web app (SPA, REST API, Sql database) with a new-user registration form. The form collects a first and last name of a new user. It is decided that the middle name needs to be collected on the form as well.

What changes need to be made to accomodate this?

• The SQL table needs to have “middle_name” column added. Requires a DDL script.
• The Java layer needs to update a USER entity to include middleName field. Java Change.
• The REST API - if using a DTO, needs the DTO updated. Java Change.
• The repository code, if implemented manually, needs to be updated. (Entity mapping can alleviate some changes). Java Change.
• The UI needs to add a field in the form to collect the middle name, and likely modify the model stored in the UI. JavaScript/Typescript change.

Also unit tests need to be updated, flyway or liquidbase scripts, redeployments have to be scheduled (if the deployment pipeline is working). Checkstyles and test coverage re-ran.

• May 02, 2023
• Batch Processing

Spring Batch: Restart a Failed Batch Job Using an API Endpoint

An key thing to remember in Spring Batch is that an instance of a Spring Batch job can only be restarted if it FAILED. If the job completeds SUCCESSFULLY a new instance of the job will have to be created to run it again.

When restarting batch jobs manually, is helpful to create an API endpoint for restarting failed batch jobs.

When a batch job is run the first time, it is assigned a batch_job_instance_id and a batch_job_execution_id.

• Apr 27, 2023
• Front End Development

Create a Service in Angular

To create a service in Angular:

cd /src/app/directory_to_create_service
ng generate service lookup 

To use service in a component inject into the constructor.

constructor(private lookupService: LookupService);
someMethod() {
    return this.lookupService.lookSomethingUp();
}

• Apr 27, 2023
• Batch Processing

Spring Batch: Chunk Size

In Spring Batch, when configuring a step you can set the chunk size.

This is a very critical and often overlooked settings.

When developing locally, it’s difficult to catch performance problems because typically local data sets are very small. However once deployed the performance problems can be crippling.

The chunk size determines how many records are processed before a commit is triggered.

So if your chunk size is set to 10 and you have 1 million records to process, the application will trigger 100,000 commits. This will be very slow.

• Apr 25, 2023
• Kubernetes

Monitor Memory and CPU of a Pod in Rancher

If you have a kubernetes pod being terminated due to OOMKilled (Out of Memory Killed), you can monitor the memory usage of the pod during the action that is using too much memory.

1. Workload -> Pods -> Find ther POD to monitor and click the name
2. Click Metrics
3. Set the range and refresh parameters (5s refresh during active monitoring)
4. Run process (trigger it through an API call/Swagger endpoint)
5. Monitor memory utilization during the process

• Apr 24, 2023
• DevOps

Create Spring App and Deploy in 8 Steps

To create a brand new spring boot app and deploy to a web server in 8 steps you can use the following:

Prerequisites

• Install Heroku CLI
• Install Spring CLI

Commands

heroku login
spring init --dependencies=web demo
cd demo
git init
git add .
git commit -m "new spring app"
heroku create
git push heroku master

• Apr 24, 2023
• Spring

Spring Boot: Install Spring Boot CLI

To Install the Spring Boot Command Line Interface:

brew tap spring-io/tap
brew install spring-boot

• Apr 14, 2023
• Kubernetes

Kubernetes Pod Dies Unexpectedly With No Error in Console Log

If you have a container in kubernetes unexpectedly disconnect with no error in the console log you can check the pod yaml for the last state of the pod.

Go to your Pod -> (three dots) Edit Yaml -> scroll to status (at the bottom) -> expand.

From here you can see the last status such as:

reason: OOMKilled

• Mar 12, 2023
• Kubernetes

Check Spring Profiles in Rancher

Spring profiles can be used to toggle behaviors in a spring app. If your app is running in a container in Rancher (Kubernetes) you may need to see what spring profiles are being applied. To check what spring profiles a service is running in Rancher:

• Click Workload in the navigation bar -> Pods
• Search for your service and click on it
• Click related resources
• Click configMap
• Scroll to SPRING_PROFILES_ACTIVE

• Mar 11, 2023
• Batch Processing

Spring Batch: Running Batch Jobs Asynchronously

When running multiple batch jobs in Spring Batch, the default JobLauncher waits to launch a job until the previous job running is COMPLETE. In order to run multiple batch jobs simultaneously, the JobLauncher must be configured to run jobs asynchronously.

Configure the async job launcher

public JobLauncher customJobLauncher(@Autowired final JobRepository jobRepository) {
    final SimpleJobLauncher jobLauncher = new SimpleJobLauncher();
    jobLauncher.setJobRepository(jobRepository);

    // Here we configure the async task executer
    jobLauncher.setTaskExecutor(new SimpleAsyncTaskExecutor());
    jobLauncher.afterPropertiesSet();
    return jobLauncher;
}

Run a job with the asyc job launcher

public class BatchJobScheduler {

    @Autowired
    private JobLauncher customJobLauncher;

    @Autowired
    private Job customJob;

    @Scheduled(cron="* * 1 * * *")
    public BatchStatus scheduleJob() {
        final JobExecution ex = customJobLauncher.runJob(customJob, getJobParameters());
        return ex.getStatus();
    }

    JobParameters getJobParameters() {
        // Get the job parameters
    }
}

• Feb 11, 2023
• Batch Processing

Spring Batch: Limit How Frequent a Batch Job Runs

Often in designing batch jobs you want to limit the frequency that a job can run. For example, if you want a batch job to run once a day, you can use @Scheduled annotation to run the batch job once a day. This is enough if you only run one instance of the batch program and there are no other ways to launch the job.

But if you have manual ways to launch the same job, or you are you using kubernetes to run multiple instances of your batch program, the batch job may be attempted to run multiple times in one day, even if you intend to only run once a day.

• Jan 11, 2023
• Batch Processing

Spring Batch: Sharing Batch Step Configurations

When defining a Spring batch step, often common configurations are added to almost every step. In order to not violate DRY (Don’t Repeat Yourself), a StepBuilder can be customized upstream with all the shared configurations.

Configuring the StepBuilder

@Configuration
public class SharedStepBuilderFactory extends StepBuilderFactory {

    @Autowired
    public JobRepository jobRepository;

    @Autowired
    public PlatformTransactionManager transactionManager;

    // Inject shared values here and use them as well
    @Value
    private String logDirectory;

    public SharedStepBuilderFactory(JobRepository jobRepository, PlatformTransactionManager transactionManager) {
        super(jobRepository, transactionManager);
    }

    @Override
    public StepBuilder get(final String name) {
        // Get the default step builder
        final StepBuilder builder = super.get(name);

        // Add listeners you want for EVERY step
        stepBuilder.listener(new StepLoggingListener());
        stepBuilder.listener(getCustomLoggingListener(logDirectory);
        return stepBuilder;
        }
}

Implementation

@Configuration
public class StepConfig {

    @Bean
    public Step getStepOne(final SharedStepBuilderFactory sharedStepBuilderFactory) {
        return sharedStepBuilderFactory
        .get("stepOne") 
        .reader(stepOneReader())
        .writer(stepOneWriter())
        .build(); // The listeners are already configured for this step from the SharedStepBuilderFactory
    }
}

• Nov 23, 2022
• Java

Java Tip: Method Reference

Instead of

for(String name: names) {
	System.out.println(name);
}

Do

// This is called a method reference
names.foreach(System.out::println);

• Sep 11, 2022
• Database

Postgres: Query Results From Today Only

When you want to query only on results from today in a Postgres database use the >= now()::date syntax.

Example

select *
from orders 
where create_time >= now()::date

• Jul 14, 2020
• Programming

Rules of Thumb for Software Developers

Fetching Data

• Remember Performance issues go unnoticed during development because the data set is too small - Vlad Mihalcea
• A transaction should fetch only as much data as required by the current executing business logic - Vlad Mihalcea

Optimization

• Premature optimization is the root of all evil - Donald E. Knuth
• Measure performance before and after each attempted optimization. J Bloch
• Programs spend 90 percent of their time in 10 percent of the code.

Design

• The components of design that are most difficult to change after the fact are those specifying interactions between components and with the outside world. Cheif among these design components are APIs, wire-level protocols, and persistent data formats. Bloch
• Strive to write good programs rather than fast ones. J Block

• Jun 20, 2020
• Command Line

Find a File From the Command Line

To find a file from the command line:

find . -type f -iname "*pattern_search_for*"

• Jul 18, 2019
• DevOps

Install Heroku CLI and Setup Heroku Autocomplete

On a Mac command line

brew tap heroku/brew && brew install heroku

Enable heroku autocomplete

heroku autocomplete
printf "$(heroku autocomplete:script bash)" >> ~/.bashrc; source ~/.bashrc

To use autocomplete type: Heroku and Tab,Tab

• Apr 20, 2019
• Git

Git: Pretty Git Graphs

In order to view git history from the command line:

git log

However if you want to view the history in a cleaner format I recommend a custom git log.

Add the following to .gitconfig (typically at ~.gitconfig)

[alias]
    lg = lg1
    lg1 = lg1-specific --all
    lg2 = lg2-specific --all

    lg1-specific = log --graph --abbrev-commit --decorate --format=format:'%C(bold blue)%h%C(reset) - %C(bold green)(%ar)%C(reset) %C(white)%s%C(reset) %C(dim white)- %an%C(reset)%C(auto)%d%C(reset)'
    lg2-specific = log --graph --abbrev-commit --decorate --format=format:'%C(bold blue)%h%C(reset) - %C(bold cyan)%aD%C(reset) %C(bold green)(%ar)%C(reset)%C(auto)%d%C(reset)%n''          %C(white)%s%C(reset) %C(dim white)- %an%C(reset)'

Now from the command line run:

• Jan 01, 0001
• Java

Java Tip: ComputeIfAbsent

Often we have to loop through a list of items and convert it into a map, with the key being some sort of category and the value being a set of the associated items ie Map<Key, Set<Object>>. The old way of doing this looped through the list and before adding checked if the key was present. Using computeIfAbsent we can clean up this syntax considerably.

Instead of

// Map of item types to items
Map<String, Set<Items>> map = new HashMap<>();

for (Item item: items) {
    if (map.containsKey(item.getType())) {
        map.get(item.getType().add(item));
    } else {
        Set newSet = new HashSet<>();
        newSet.add(item);
        map.put(item.getType(), newSet));
    }
}

Do

// Map of item types to items
Map<String, Set<Items>> map = new HashMap<>();

for (Item item: items) {
    map.computeIfAbsent(item.getType(), v -> new HashSet<>()).
    add(item);
}

• Jan 01, 0001
• Batch Processing

Spring Batch: Query All the Steps of a Batch Job

In Spring Batch, in order to get the job_execution_id of the last batch job instance for a given batch job name use this query:

select bje.job_execution_id from batch_job_instance bji
join batch_job_execution bje
on bji.job_instance_id = bje.job_instance_id
where bji.job_name = 'jobname'
order by bje.start_time desc
limit 1;

In order to get all the steps for the latest batch job instance for a given batch job name use this query:

select *
from batch_job_execution bje
join batch_job_instance bji
on bje.job_instance_id = bji.job_instance_id
join batch_job_step_execution bse
on bse.job_execution_id = bje.job_execution_id
and bje.job_execution_id = 
(
    select bje.job_execution_id from batch_job_instance bji
    join batch_job_execution bje
    on bji.job_instance_id = bje.job_instance_id
    where bji.job_name = 'jobname'
    order by bje.start_time desc
    limit 1
)
order by bse.start_time;