Withdrawal of the Class 40s started in 1976, when six locomotives were taken out of service. At over 130 tons the class were by then considered underpowered for their size. In addition, some were found to be suffering from fractures of their bogies and spares were needed to keep other class members in traffic. Many members of the class were not fitted with train air braking, leaving them unable to haul more modern freight and passenger vehicles.
The pace of non-air brake fitted locomotives being withdrawn began to increase and by 1981 the remaining locomotives were concentrated in the London Midland region of BR. Numbers dwindled slowly until by the end of 1984 there were only thirteen class members still running. Included in this number was pioneer locomotive 40122 (previously D200), which having been withdrawn in 1981, was re-instated in mid-1983 and painted in its original green livery to haul railtours.
In addition, four locomotives were temporarily returned to service as Class 97 departmental locomotives, numbered 97405-97408. They were used to work engineering trains for the remodelling project at Crewe station. These were withdrawn by early 1987. 40122 was eventually withdrawn in 1988 and presented to the National Railway Museum. Six other locomotives were also preserved.