Edward Coles

Edward Coles (December 15, 1786 – July 7, 1868) was an American planter and politician, elected as the second Governor of Illinois (1822 to 1826). From an old Virginia family, Coles as a young man was a neighbor and associate of presidents Thomas Jefferson and James Monroe, as well as, secretary to President James Madison from 1810 to 1815.

An anti-slavery advocate throughout his adult life, Coles inherited a plantation and slaves but eventually left Virginia for the Illinois Territory to set his slaves free. He manumitted 19 slaves in 1819 and acquired land for them. In Illinois, he first participated in a campaign to block extending existing slavery in the new state, and then two years later at his inauguration as Governor, he called for the end of slavery in Illinois altogether, which was later achieved. Coles corresponded with and advised both Jefferson and Madison to free their slaves, and publicly supported abolition. In his final years in Philadelphia, Pennsylvania, he helped shaped early historians' views of the presidents' republican ideals.

Early life and education
Coles was born on December 15, 1786, at Enniscorthy, a plantation in central Virginia's Albemarle County on the Hardware River, a tributary of the James River. He was the youngest male among ten surviving children of John Coles (1745–1808) and Rebecca Tucker (1750–1826). Young Coles' earliest teachers were prominent lawyer Wilson Cary Nicholas and Mr. (probably Rev.) White who lived by Dyer's Store. After a term at Hampden-Sydney College in Hampden-Sydney, Virginia, Coles transferred to the College of William and Mary in Williamsburg, Virginia.

While at William and Mary, Coles was strongly influenced by the enlightenment ideals taught by the Rt. Rev. James Madison (first Episcopal Bishop of the Diocese of Virginia and President of the College). The teacher and cleric considered slavery morally indefensible, but a problem without a clear solution. Young Coles determined not to be a slaveholder and not to live where slavery was accepted. However, he kept these views from his father, whose illness (and that of his elder brother) caused Coles to end his formal education in the summer of 1807, for fear that his father would substitute other property for slaves when writing his last will and testament. His bachelor uncles in Norfolk, Travis and John Tucker, had freed slaves when such had become legal in Virginia, and Coles' father John noted that some of the slaves freed by Travis (a devout Methodist) were now living in near starvation. Keeping quiet thus ensured that Coles would inherit slaves, thus providing him with the opportunity to give freedom.

When his father died in 1808, Coles received 12 slaves and a 782-acre plantation on the Rockfish River in Nelson County, Virginia, subject to a mortgage. After John Coles' estate was settled on Christmas Eve, 1808, Edward Coles revealed his emancipation plans to his family, to great consternation. As he sorted through the challenges posed by family resistance and Virginia law (which since 1806 required freed slaves to leave the state within a year, and had also increased restrictions on already free blacks), Coles abandoned his earliest plan to free his slaves in Virginia. He went to Kentucky in the summer of 1809 to investigate a land claim of his uncle Travis Tucker, but came home without plans to move to that new state (which allowed slavery).

Coles placed his plantation for sale in December, 1809, despite the collapsed real estate market during the depression of 1807, and began to plan for a move to the Northwest Territory (where slavery had been at least technically abolished in 1787). However, for years he received no reasonable offers, and so continued to operate it through an overseer. Coles turned down offers to exchange his slaves for other property, but honored the requests of his family and neighbors to keep his plans secret from his slaves.

Family
The Coles family was one of the First Families of Virginia. His great-grandfather, Walter Coles, had been a customs officer in Enniscorthy, Ireland and died there in 1640. His grandfather John had been one of the petitioners requesting for Richmond to be recognized as a new town, and he continued to develop the family's business and social ties through marriage to the youngest daughter of Quaker merchant Isaac Winston. Edward Coles's father, John, or John II, developed Enniscorthy from a hunting camp into a profitable farm, and continued the family's business and social success.

Edward Coles's maternal grandfather was born in Bermuda, and related to Virginia jurist St. George Tucker. His mother's maternal ancestors were among the "first & most respectable settlers at old Jamestown."

Edward's elder brother, Isaac A. Coles, served as private secretary to both Thomas Jefferson and James Madison during their administrations. Thomas Jefferson's Monticello plantation was nearby in Albemarle County. Furthermore, the wife of James Madison, Dolley Payne Todd Madison, was Coles's first cousin, and Coles became a frequent guest at their Montpelier plantation also nearby. James Monroe owned Ash Lawn-Highland plantation on the other side of Green Mountain for 24 years (until forced to sell it in 1825 due to financial problems), and offered young Coles use of his library, although the relationship with this family was more distant since Monroe split his time at Oak Hill plantation in Loudoun County.

Isaac managed and ultimately inherited Enniscorthy, subject to a life estate held by his mother, who died in 1826. His brother Walter had drawn his share of the inheritance early and managed Woodville plantation for many years before his father's death. John Coles III built a mansion called Estouteville on his inherited portion, and Tucker Coles built Tallwood plantation on the upper acres that he inherited—both married daughters of Sir Peyton Skipwith, the only baronet in Virginia. Their sister Rebecca became the second wife of South Carolina planter Richard Singleton, with whom she had five children. Her sister Elizabeth (Betsy) never married. Mary Coles married Robert Carter and moved to his nearby Redlands plantation. Their sister Emily married Richmond lawyer John Rutherfoord, who owned Tuckahoe plantation in Goochland County and later served as acting governor of Virginia (1841–42). Their son John Coles Rutherfoord witnessed the Civil War's devastation, and his grandson W.A.R. Goodwin became an Episcopal minister and helped found Colonial Williamsburg. Their sister Callie (Sarah) married Andrew Stevenson, who served as Speaker of the United States House of Representatives as well as American minister to the United Kingdom.

Service in the White House
Some months after taking office as President, James Madison invited Coles to become his private secretary. His brother Isaac had been performing those duties, as well as delicate courier tasks (particularly with France in the days leading up to the War of 1812). However, Isaac Coles, perhaps inappropriately acting out his earlier military career dreams, beat up Maryland representative Roger Nelson and, after a critical Congressional report concerning the incident, was forced to submit his resignation on December 29, 1809. Neighbor James Monroe (soon to become Secretary of State) convinced Edward Coles to accept that secretarial position, and Coles served from January 1810 to March 1815, despite intervals of ill-health.

Coles' term as presidential secretary delayed his plans to free his slaves. However, Coles developed a good relationship with Madison, with whom he would often speak with "perfect candor", and formed a lasting admiration for the president. While Madison's secretary, Coles initiated private correspondence with Thomas Jefferson over the issue of slavery, as discussed below. Coles gained political experience as Madison's assistant, served as his primary emissary to Congress, and managed much of the patronage flowing from the 162-employee executive branch. Among other duties, Coles hand-copied the president's official correspondence for the national archives.

Coles met John Adams during a tour of north-eastern states in 1811. Along with Benjamin Rush, Coles worked to diminish tensions and rewarm relations between Adams and Thomas Jefferson. Coles also spent considerable time in Philadelphia receiving medical treatment from Dr. Physick, among others, as well as began a long friendship with Nicholas Biddle, who became controversial as a banker and anti-slavery advocate. As the War of 1812 ended, Coles resigned due to continued ill health in February 1815.

Upon recovering in June, Coles and his 40-year-old mulatto slave and coachman, Ralph Crawford, toured the Northwest Territory in search of land Coles could purchase and develop as a home for himself and a place for the slaves he still proposed to free. Coles wrote letters to President Madison and to relatives expressing dissatisfaction with the high land prices in Ohio, then with squatters, real estate speculators and fraudulent businessmen as he travelled further west into the Indiana (and what became the Illinois) Territory. In the Missouri Territory Coles bought some land for investment, before finally embarking in St. Louis on a keel boat for his trip down the Mississippi River to New Orleans, from where Coles ultimately sailed home to Virginia.

Coles was delayed again in fulfilling his covenant with freedom by a diplomatic trip to Russia (1816–1817) at President Madison's request, to resolve a diplomatic incident concerning the Russian consul's arrest in Philadelphia for sexually assaulting a maid. After successfully completing his mission in St. Petersburg, Coles toured Brussels, Paris and England. While in England, Coles met ambassador John Quincy Adams and social reformer Morris Birkbeck. Coles extolled such enthusiasm about Illinois that Birkbeck bought land, moved and established a settlement. Upon his return, Coles wrote a paper comparing slavery and Russian serfdom, so if an ulterior motive for the diplomatic assignment was dissuading Coles from his manumission plan, it failed. Moreover, Coles wrote about (and long remembered) the pervasive bribery and unethical business practices he encountered in Russia.

Correspondence with Jefferson
In 1814 Coles wrote a letter to his Albemarle County neighbor Thomas Jefferson, asking the former President to again embark on a campaign of emancipation and publicly work for an end to slavery in Virginia. Jefferson's response has become a signal document in the study of Jefferson's troubling and complex relationship with the institution of slavery. At age 71 and generally retired from politics and because Virginia law did not allow for emancipation of slaves, Jefferson declined Coles' request, advising his young friend and associate to stay in Virginia to help in the long-term demise of slavery. Coles’ disappointment is clear in his return letter of September 26, 1814.

Edward Coles to Thomas Jefferson, July 31, 1814

Dear Sir,

I never took up my pen with more hesitation or felt more embarrassment than I now do in addressing you on the subject of this letter. The fear of appearing presumptuous distresses me, and would deter me from venturing thus to call your attention to a subject of such magnitude, and so beset with difficulties, as that of a general emancipation of the Slaves of Virginia, had I not the highest opinion of your goodness and liberality, in not only excusing me for the liberty I take, but in justly appreciating my motives in doing so.

I will not enter on the right which man has to enslave his Brother man, nor upon the moral and political effects of Slavery on individuals or on Society; because these things are better understood by you than by me. My object is to entreat and beseech you to exert your knowledge and influence, in devising, and getting into operation, some plan for the gradual emancipation of Slavery. This difficult task could be less exceptionably, and more successfully performed by the revered Fathers of all our political and social blessings, than by any succeeding statesmen; and would seem to come with peculiar propriety and force from those whose valor wisdom and virtue have done so much in meliorating the condition of mankind. And it is a duty, as I conceive, that devolves particularly on you, from your known philosophical and enlarged view of subjects, and from the principles you have professed and practiced through a long and useful life, preeminently distinguished, as well by being foremost in establishing on the broadest basis the rights of man, and the liberty and independence of your Country, as in being throughout honored with the most important trusts by your fellow-citizens, whose confidence and love you have carried with you into the shades of old age and retirement. In the calm of this retirement you might, most beneficially to society, and with much addition to your own fame, avail yourself of that love and confidence to put into complete practice those hallowed principles contained in that renowned Declaration, of which you were the immortal author, and on which we bottomed our right to resist oppression, and establish our freedom and independence.

I hope that the fear of failing, at this time, will have no influence in preventing you from employing your pen to eradicate this most degrading feature of British Colonial policy, which is still permitted to exist, notwithstanding its repugnance as well to the principles of our revolution as to our free Institutions. For however highly prized and influential your opinions may now be, they will be still much more so when you shall have been snatched from us by the course of nature. If therefore your attempt should now fail to rectify this unfortunate evil—an evil most injurious both to the oppressed and to the oppressor—at some future day when your memory will be consecrated by a grateful posterity, what influence, irresistible influence will the opinions and writings of Thomas Jefferson have on all questions connected with the rights of man, and of that policy which will be the creed of your disciples. Permit me then, my dear Sir, again to intreat you to exert your great powers of mind and influence, and to employ some of your present leisure, in devising a mode to liberate one half of our Fellowbeings from an ignominious bondage to the other; either by making an immediate attempt to put in train a plan to commence this goodly work, or to leave human Nature the invaluable Testament—which you are so capable of doing—how best to establish its rights: So that the weight of your opinion may be on the side of emancipation when that question shall be agitated, and that it will be sooner or later is most certain—That it may be soon is my most ardent prayer—that it will be rests with you.

I will only add, as an excuse for the liberty I take in addressing you on this subject, which is so particularly interesting to me; that from the time I was capable of reflecting on the nature of political society, and of the rights appertaining to Man, I have not only been principled against Slavery, but have had feelings so repugnant to it, as to decide me not to hold them; which decision has forced me to leave my native state, and with it all my relations and friends. This I hope will be deemed by you some excuse for the liberty of this intrusion, of which I gladly avail myself to assure you of the very great respect and esteem with which I am, my dear Sir, your every sincere and devoted friend.


 * Edward Coles

Thomas Jefferson to Edward Coles August 25, 1814 Monticello

DEAR SIR,-- Your favour of July 31, was duly received, and was read with peculiar pleasure. The sentiments breathed through the whole do honor to both the head and heart of the writer. Mine on the subject of slavery of negroes have long since been in possession of the public, and time has only served to give them stronger root. The love of justice and the love of country plead equally the cause of these people, and it is a moral reproach to us that they should have pleaded it so long in vain, and should have produced not a single effort, nay I fear not much serious willingness to relieve them & ourselves from our present condition of moral & political reprobation. From those of the former generation who were in the fulness of age when I came into public life, which was while our controversy with England was on paper only, I soon saw that nothing was to be hoped. Nursed and educated in the daily habit of seeing the degraded condition, both bodily and mental, of those unfortunate beings, not reflecting that that degradation was very much the work of themselves & their fathers, few minds have yet doubted but that they were as legitimate subjects of property as their horses and cattle. The quiet and monotonous course of colonial life has been disturbed by no alarm, and little reflection on the value of liberty. And when alarm was taken at an enterprize on their own, it was not easy to carry them to the whole length of the principles which they invoked for themselves. In the first or second session of the Legislature after I became a member, I drew to this subject the attention of Col. Bland, one of the oldest, ablest, & most respected members, and he undertook to move for certain moderate extensions of the protection of the laws to these people. I seconded his motion, and, as a younger member, was more spared in the debate; but he was denounced as an enemy of his country, & was treated with the grossest indecorum. From an early stage of our revolution other & more distant duties were assigned to me, so that from that time till my return from Europe in 1789, and I may say till I returned to reside at home in 1809, I had little opportunity of knowing the progress of public sentiment here on this subject. I had always hoped that the younger generation receiving their early impressions after the flame of liberty had been kindled in every breast, & had become as it were the vital spirit of every American, that the generous temperament of youth, analogous to the motion of their blood, and above the suggestions of avarice, would have sympathized with oppression wherever found, and proved their love of liberty beyond their own share of it. But my intercourse with them, since my return has not been sufficient to ascertain that they had made towards this point the progress I had hoped. Your solitary but welcome voice is the first which has brought this sound to my ear; and I have considered the general silence which prevails on this subject as indicating an apathy unfavorable to every hope. Yet the hour of emancipation is advancing, in the march of time. It will come; and whether brought on by the generous energy of our own minds; or by the bloody process of St Domingo, excited and conducted by the power of our present enemy, if once stationed permanently within our Country, and offering asylum & arms to the oppressed, is a leaf of our history not yet turned over. As to the method by which this difficult work is to be effected, if permitted to be done by ourselves, I have seen no proposition so expedient on the whole, as that as emancipation of those born after a given day, and of their education and expatriation after a given age. This would give time for a gradual extinction of that species of labour & substitution of another, and lessen the severity of the shock which an operation so fundamental cannot fail to produce. For men probably of any color, but of this color we know, brought from their infancy without necessity for thought or forecast, are by their habits rendered as incapable as children of taking care of themselves, and are extinguished promptly wherever industry is necessary for raising young. In the meantime they are pests in society by their idleness, and the depredations to which this leads them. Their amalgamation with the other color produces a degradation to which no lover of his country, no lover of excellence in the human character can innocently consent. I am sensible of the partialities with which you have looked towards me as the person who should undertake this salutary but arduous work. But this, my dear sir, is like bidding old Priam to buckle the armour of Hector "trementibus aequo humeris et inutile ferruncingi." No, I have overlived the generation with which mutual labors & perils begat mutual confidence and influence. This enterprise is for the young; for those who can follow it up, and bear it through to its consummation. It shall have all my prayers, & these are the only weapons of an old man. But in the meantime are you right in abandoning this property, and your country with it? I think not. My opinion has ever been that, until more can be done for them, we should endeavor, with those whom fortune has thrown on our hands, to feed and clothe them well, protect them from all ill usage, require such reasonable labor only as is performed voluntarily by freemen, & be led by no repugnancies to abdicate them, and our duties to them. The laws do not permit us to turn them loose, if that were for their good: and to commute them for other property is to commit them to those whose usage of them we cannot control. I hope then, my dear sir, you will reconcile yourself to your country and its unfortunate condition; that you will not lessen its stock of sound disposition by withdrawing your portion from the mass. That, on the contrary you will come forward in the public councils, become the missionary of this doctrine truly Christian; insinuate & inculcate it softly but steadily, through the medium of writing and conversation; associate others in your labors, and when the phalanx is formed, bring on and press the proposition perseveringly until its accomplishment. It is an encouraging observation that no good measure was ever proposed, which, if duly pursued, failed to prevail in the end. We have proof of this in the history of the endeavors in the English parliament to suppress that very trade which brought this evil on us. And you will be supported by the religious precept, "be not weary in well-doing." That your success may be as speedy & complete, as it will be of honorable & immortal consolation to yourself, I shall as fervently and sincerely pray as I assure you of my great friendship and respect.


 * Thomas Jefferson

Edward Coles to Thomas Jefferson September 26, 1814 Washington Dear Sir,

I must be permitted again to trouble you, my dear Sir, to return my grateful thanks for the respectful and friendly attention shown to my letter in your answer of the 25th ulto. Your favorable reception of sentiments not generally avowed if felt by our Countrymen, but which have ever been so inseparably interwoven with my opinions and feelings as to become as it were the rudder that shapes my course even against a strong tide of interest and of local partialities, could not but be in the highest degree gratifying to me. And your interesting and highly prized letter, conveying them to me in such flattering terms, would have been called forth my acknowledgments before this but for its having been forwarded to me to the Springs, and from thence it was again returned here before I received it, which was only a few days since.

Your indulgent treatment encourages me to add—that I feel very sensibly the force of your remarks on the impropriety of yielding to my repugnancies in abandoning my property in Slaves and my native State. I certainly should never have been inclined to yield to them if I had supposed myself capable of being instrumental in bringing about a liberation, or that I could by my example meliorate the condition of these oppressed people. If I could be convinced of being in the slightest degree useful in doing either, it would afford me very great happiness, and the more so as it would enable me to gratify many partialities by remaining in Virginia. But never having flattered myself with the hope of being able to contribute to either, I have long since determined, and should, but for my bad health ere this, have removed, carrying along with me those who had been my Slaves, to the Country North West of the river Ohio.

Your prayers I trust will not only be heard with indulgence in Heaven, but with influence on earth. But I cannot agree with you that they are the only weapons of one at your age, nor that the difficult work of cleansing the escutchion of Virginia of the foul stain of slavery can best be done by the young. To effect so great and difficult an object great and extensive powers both of mind and influence are required, which can never be possessed in so great a degree by the young as by the old. And among the few of the former who might unite the disposition with the re-quisite capacity, they are too often led by ambitious views to go with the current of popular feeing, rather than to mark out a course for themselves, where they might be buffetted by the waves of opposition; and indeed it is feared these waves would in this case be too strong to be effectually resisted, by any but those who had gained by a previous course of useful employment the firmest footing in the confidence and attachment of their Country. It is with them, therefore, I am persuaded, that the subject of emancipation must originate; for they are the only persons who have it in their power effectually to arouse and enlighten the public sentiment, which in matters of this kind ought not to be expected to lead but to be led; nor ought it to be wondered at that there should prevail a degree of apathy with the general mass of mankind, where a mere passive principle of right has to contend against the weighty influence of habit and interest. On such a question there will always exist in society a kind of vis inertia, to arouse and overcome which require a strong impulse, which can only be given by those who have acquired a great weight of character, and on whom there devolves in this case a most solemn obligation. It was under these impressions that I looked to you, my dear sir, as the first of our aged worthies, to awaken our fellow Citizens from their infatuation to a proper sense of Justice and to the true interest of their country, and by proposing a system for the gradual emancipation of our Slaves, at once to form a rallying point for its friends, who enlightened by your wisdom and experience, and supported and encouraged by your sanction and patronage, might look forward to a propitious and happy result. Your time of life I had not considered as an obstacle to the undertaking. Doctor Franklin, to whom, by the way, Pennsylvania owes her early riddance of the evils of Slavery, was as actively and as usefully employed on as arduous duties after he had past your age as he had ever been at any period of his life.

With apologizing for having give you so much trouble on this subject, and again repeating my thanks for the respectful and flattering attention you have been pleased to pay to it, I renew the assurances of the great respect and regard which makes me most sincerely.


 * Yours


 * Edward Coles

Illinois
In the fall of 1817, Coles sold his plantation to his eldest brother Walter, having declined James Monroe's request that he continue as the new President's private secretary. Instead, Coles embarked on a second reconnaissance mission to the Northwest Territories (1818). He bought land in the American Bottom in Illinois Territory. Coles also participated in the Illinois Constitutional Convention at Kaskaskia, Illinois after Indiana became a state. Coles worked with Baptist John Mason Peck, Methodist Peter Cartwright, Quaker James Lemen, and publisher Hooper Warren to successfully oppose a faction that wanted to legitimize slavery in the new territory's constitution.

Coles then returned to Virginia, planning to display his deep moral objections to slavery and finally manumit the slaves he inherited from his father after leaving the Commonwealth. In late March 1819, having collected the final payment from Walter, Edward Coles was ready to move to the Illinois Territory. President Monroe had appointed him Register of Lands for the new territory, with an office at Edwardsville.

Coles sent his trusted slave (and travel companion during his previous Northwest Territory trips) Ralph Crawford with wagons and 16 other slaves (total 6 adults and 11 children) ahead on the Great Wagon Road north to Pennsylvania. To the derision of many family and friends, Coles had let the slaves ride on ahead, none of them knowing his plans to free them at that time. Coles traveled separately. They met at Brownsville, Pennsylvania, where the party boarded a pair of flatboats and began a water-bound journey: floating on the Monongahela River north to Pittsburgh, then west along the Ohio River toward Illinois. Coles selected a point west of Pittsburgh to announce to his slaves their immediate freedom and also his plan to provide land to each head of a family. Coles captured the scene in an autobiographical piece written some 25 years later. Decades later, the river emancipation became the subject of a mural in the first floor (south hall) of the Illinois State Capitol.

The Coles party arrived in Edwardsville, early in May 1819, and Coles began his service as Register of Lands. He also completed the manumission process by purchasing land so as to give each freed head of family 160 acre. Coles also provided employment and other ongoing support for those he had freed. As Register through 1822, Coles mediated and untangled complicated land disputes, thus earning a reputation for fairness and honesty.

Term as Illinois Governor
Coles ran for governor in the election of 1822. To his great surprise, he won the election by a very tight margin, defeating Chief Justice Joseph Phillips (an ally of Judge Jesse Thomas and prominent slavery advocate in his own right who eventually returned to Kentucky), Associate Justice Thomas C. Browne (the eventual candidate of the Ninian Edwards faction) and militia commander James B. Moore. Coles had left Illinois on election day believing he had lost, and received the news of his victory while in Virginia recovering from bilious fever. He accordingly cleared up his land office accounts in Washington, D.C. and returned to Illinois. Madison sent him a package containing a pedometer, and a note, "As you are about to assume new motives to walk in a straight path, and with measured steps, I wish you to accept the little article enclosed, as a type of the course I am sure you will pursue, and as a token of the affection I have so long cherished for you."

Coles’ inaugural address included a clear call for the end of slavery in Illinois and revision of the Black Code, as well as advocated internal improvements (especially a canal link to the Great Lakes) and aid to agriculture and education. Slavery was a very important topic at the time, because that first state constitution only permitted current practices using slave labor in the salines (salt evaporation factories) through 1825. A proslavery faction had hoped to eliminate the first constitution's anti-slavery clause and transform Illinois into a slave state like Missouri. Coles’ bold call for an end to slavery stiffened their resolve and led to a rancorous legislative effort which began with the Shaw-Hansen Affair (concerning whether to seat a pro-slavery candidate backed by voters who came from Missouri, or his anti-slavery opponent). Governor Edward Coles led the opposition to a bill approving a referendum to hold another constitutional convention, recognizing it as a dishonest attempt to more clearly legalize slavery in the state. After the bill passed, Coles committed his total pay as governor ($4000) to defeat the referendum, and led a committee of anti-slavery citizens, religious leaders, and legislators (who committed another $1000). The aristocratic, awkward Virginian and his allies then dispelled a plethora of false economic arguments spread by slavery proponents, while in the aftermath of the Panic of 1819 keeping secret their printing help from much-reviled Philadelphians (Nicholas Biddle and Roberts Vaux). The 18-month political struggle used committees in each county as well as traveling preachers.

On August 2, 1824, Illinois voters rejected the pro-slavery convention referendum (as well as re-elected anti-slavery U.S. representative Daniel Pope Cook). However, by year's end pro-slavery legislators refused to approve Coles' appointment of his anti-slavery friend Morris Birkbeck as secretary of state. Furthermore, lieutenant governor Adolphus Hubbard attempted to wrest the governorship away from Coles during his trip to Virginia in late 1825, causing additional confusion, although Hubbard also lost the gubernatorial election the next year to Ninian Edwards (the state Constitution also included a provision modelled on Virginia's which precluded governors from running for reelection). Finally, a lawsuit that political opponents in Madison County, Illinois brought against Coles for failing to pay a slave tax on his freed slaves years earlier took several more years, including shenanigans by pro-slavery judge Samuel McRoberts, before the Illinois Supreme Court ruled such payment unnecessary. Coles' valedictory speech as governor in December 1826 reminded legislators of his previous speeches urging them to abolish slavery and its remnants (especially heritability) in the new state, as well as to finance a canal to the Great Lakes watershed and a penitentiary.

After his term as governor expired, Coles returned briefly to Virginia, where his mother had died that spring. Coles then returned to his farm outside of Edwardsville. He focused on agricultural and business pursuits, between continued trips to Virginia and Philadelphia to visit family and friends and to search for a wife. His friend Daniel Pope Cook was defeated by a proslavery opponent in 1826, and legislators had selected their pro-slavery speaker John McLean to fill unexpired U.S. Senate terms in 1824 and 1829 (voters ultimately electing Elias Kent Kane and John McCracken Robinson to those seats). Coles made his last run for public office in 1831. As candidate for Congress, running against eight candidates including pro-slavery Democrats Joseph Duncan (allied with Treasury Secretary William H. Crawford) and Sidney Breese (a Jacksonian Democrat), Coles polled a distant third. He had been out of public view for some years, and refused to align himself with any political party. Instead, Coles campaigned on his successful term as governor, proclaimed his association with the founding fathers, and criticized the Jacksonian platform. Still, Coles felt devastated by the political defeat, and moved back east. Nonetheless, in 1835, Illinois legislators authorized Coles to sell bonds to finance his canal project, but since they also refused to back the bonds with state credit, sales proved slow.

Back to Virginia
Worried about his unmarried status and increasing partisanship, Coles decided to leave Illinois shortly after his election loss. He made another trip to Virginia, which was involved in its own debate over slavery after Nat Turner's rebellion. After Turner's execution, Coles wrote Thomas Jefferson Randolph urging emancipation and colonization to prevent further disasters, stressing that slavery restricted Virginia's economic development. At year's end, while visiting James and Dolley Madison at Montpelier, Madison confided in Coles his wish to manumit his own slaves and asked Coles about his experiences as he tried to find the right way to accomplish this while still providing for Dolley as his widow. However, Madison died in 1836 without freeing any of his slaves which were left in a will to his wife Dolley.

Later life in Philadelphia
Coles moved to Philadelphia in 1832, gratified by its active social and intellectual life, as well as slavery's absence. At age 46, Coles married prominent socialite Sally Logan Roberts (1809 to 1883) on November 28, 1833. The couple had three children: Mary Coles, Edward Coles, Jr., and Roberts Coles. Sally Coles inherited significant property upon her father's death, but it (and much of Coles' own fortune) was devastated by the Panic of 1837. Furthermore, his family's renovated Enniscorthy plantation burned down in 1839, and his elder brother Isaac only survived the disasters by two years. Coles's last public appointment was in 1841, when he served on a committee investigating the U.S. Bank, which ultimately led to the resignation of his friend Nicholas Biddle. Coles unsuccessfully sought political appointments from his Virginia classmates who had become high federal officials, John Tyler and Winfield Scott. Still, rental income from real estate investments (widely spread geographically) kept the growing family financially comfortable. Coles and his young family travelled often to visit his extended family and properties in Virginia and Washington, D.C., as well as Illinois and later Schooley's Mountain, New Jersey. He was elected as a member of the American Philosophical Society in 1839.

However, Coles never resumed his political career, uncomfortable with the new party system. Nonetheless, he rejoiced when Abraham Lincoln, whom he remembered as a young Illinois lawyer, was elected president. The elderly ex-governor briefly met the newly elected president on his journey to Washington. Coles had also publicly taken exception to Senator Stephen A. Douglas' characterization of slavery's history in Illinois during the Kansas-Nebraska debate of 1854.

Coles turned to history during his later years. He was recognized as one of the few remaining men with close personal knowledge of both Madison and Jefferson, and burnished their reputations as champions of the republican ideals that had also motivated Coles during his entire life. Coles had lobbied both Jefferson and Madison (and later Thomas Jefferson Randolph) to free their slaves. Coles was surprised when Madison failed to do so, only later learning that lawyer Robert Taylor had persuaded the former President to leave emancipation instructions for his widow, whose father had gone bankrupt after freeing his slaves many years earlier. Dolley, with other personal economic priorities (especially a son addicted to gambling), freed no slaves upon her death in 1849. Coles also wrote about the Northwest Ordinance. However, most of Coles' own papers from Illinois were destroyed in a fire in 1852 while his friend John Mason Peck was writing a history of the new state. Coles also assisted Virginia historians William Cabell Rives and Hugh Blair Grigsby, and New Yorker Henry S. Randall.

To his father's great disappointment, Roberts Coles returned to Virginia in 1860, where he became a slaveowner (buying a plantation from a relative) and engaged to Jennie Fairfax of Richmond. After the Civil War began, he enlisted in the Green Mountain Greys and was elected captain. Roberts Coles died (as did the other Confederate captain) during the Battle of Roanoke Island, February 8, 1862.

Death
Coles died, aged eighty one years, in his home (1303 Spruce Street in Philadelphia) on July 7, 1868. While devastated by his younger son's death fighting for the Confederates, Coles lived to see slavery abolished through President Lincoln's issuing the Emancipation Proclamation as well as ratification of the Thirteenth Amendment. Coles also lived to see his elder son and namesake, Edward Coles, Jr., marry Elizabeth (Bessie) Mason Campbell, relative of anti-slavery founding father George Mason (descendant of his younger brother Thomson Mason) on February 25, 1868. Edward Coles Jr. became a prominent lawyer in Philadelphia, and a long-time vestry member of Christ Church, which still displays a plaque honoring his service on the south wall inside the nave.

The Coles family grave at Woodlands Cemetery in west Philadelphia includes the final resting places of the former governor, his wife Sally, as well as their three children. His daughter Mary Coles (who never married) and Caroline Sanford founded the Church Training school for Deaconesses in Philadelphia in 1891, and served as its president as well as a faculty member. The school generally graduated 7 to 10 pupils per year (the highest number in residence was 30 in 1912-13), which were in great demand and served not only in hospitals and parishes in Pennsylvania, but throughout the United States and missions abroad. In 1895 Mary Coles secured the re-interment of her brother Roberts from the family cemetery at Enniscorthy. Edward Jr. and Bessie Coles were also buried at the new family gravesite.

Legacy
Coles was among the very few slaveholders who manumitted his slaves entirely as a testament to the republican ethos that was at the heart of the American Revolution and enlightenment. He is also noteworthy for his attempts to pressure Thomas Jefferson and Thomas Jefferson Randolph to work to end of slavery in Virginia and for James Madison to free his slaves.

Coles County, Illinois was named for him, and more recently an elementary school on the south side of Chicago. During his lifetime, the Illinois legislature named the county seat of then-huge Pike County, Coles Grove. Coles Grove was renamed Gilead when it became the county seat of Calhoun County when Calhoun County was divided from Pike County. While a sign remains to signify Gilead as the now former-county seat of Calhoun County, there is very little remaining of the community. The county seat of Calhoun County was moved to Hardin, Illinois in 1847.

The Governor Coles State Memorial is located in Edwardsville, Illinois. The Illinois Human Rights Commission also offers a scholarship to law students in honor of the former governor.

Original sources

 * Coles, Edward, (1856). History of the Ordinance of 1787, ( primary source ) Historical Society of Pennsylvania, pp. 33, Url
 * The Coles Family papers, containing correspondence, various papers and materials belonging to Edward Coles, are available for research use at the Historical Society of Pennsylvania.