The first publicly available record contained the BBC Language course shown here and was released in 1963 - 4 years before the main catalogue started in 1967.
Originally these language records were only available direct from the BBC in a thick brown cardboard sleeve. They later became more mainstream as popularity of learning foreign languages increased.
The first release from the "main" catalogue was the radio show about our current knowledge of the universe. This was originally released in limited numbers without a picture sleeve.
This was later re-released with a picture sleeve as shown in this picture.
The first BBC single followed around three years after the above album. It may have originally only been released as a one off and the next two singles on the label didn't get full releases.
A year later another similar single was released and the idea after this continued for nearly 30 years.