In a cost-based supply chain delivery performance model, accurately depicting the penalty cost associated with an untimely delivery is an important component of the supply chain’s delivery performance. This paper introduces a two-stage time-sensitive supply chain, where the penalty cost caused by early and late delivery is treated as a power function of the default time. This default time is the deviation in time from the actual delivery time to the delivery window. Both the buyer and the supplier want to reduce the default time. After considering the product’s value over time and the buyers’ attitude with respect to a breach of contract, the penalty cost is constructed as the product of a nonlinear function of default time and as a linear function of the penalty factor. This yields the conditions associated with the optimal delivery window, to minimize the expected penalty with the power function. The paper shows the effect of the delivery window, penalty factor, and time sensitivity factor on supply chain performance, leading to recommendations for improved performance. Numerical results are provided to demonstrate the applicability of the proposed model. The model helps buyers more accurately determine costly losses, enabling them to reduce the impact of any default caused by suppliers. For exponentially distributed lead time, choosing the optimal delivery window position for delivery can reduce the expected penalty cost by as much as seven times. The average delivery time is shortened from 30 days to 20 days, and the expected penalty cost will be reduced by about 50%. This is a powerful tool for use in improving supply chain performance and also provides directions for improving strategies and a theoretical basis for buyers and suppliers wanting to jointly optimize supply chain performance.