A riding habit was the costume or dress ladies wore when riding. I don't think I can put it more simply than that. :o)
The description for the fashion plate (shown here) from the June 1812 issue of La Belle Assemblée reads as follows:
"An habit of bright green, ornamented down the front, and embroidered at the cuffs à-la-militaire with black. Small riding hat of black beaver, fancifully adorned with gold cordon and tassels, with a long ostrich feather of green in front; or a green hat with black tassels and black feather. Black half boots, laced and fringed with green. York tan gloves. When this dress is worn as a curicle or walking costume, it is made as a pelisse without the riding jacket, and confined round the waist by a belt of black and green."*
The fashion evolved as all fashions do. The picture below shows the evolution of the riding costume from 1800-1842.**
A split riding skirt, though used in the middle ages by some women of note, was not acceptable riding dress in Regency Britain. This type of riding costume was not accepted until the early 1900s.
I suppose if one were to write a Regency heroine who flouted convention in every way she could, wearing a split skirt and riding astride would be a definite way to do just that. If the heroine cares at all for her reputation, however, she would never don such indecent garb.
*La Belle Assemblée, June 1812 (Google book)
**Historic Dress in America, 1800-1870 by Elisabeth McClellan, 1910 (Google Book)