Molecular Docking of Phytochemicals: Step-by-Step Guide

To dock phytochemicals, first pick a protein target with a known binding site, then build a compound list from a curated source such as IMPPAT 2.0, Dr. Duke’s or COCONUT. Resolve every compound to a PubChem CID, download 3D structures, correct protonation at pH 7.4, convert to PDBQT, and dock with AutoDock Vina.

The hardest part of a phytochemical docking project is not the docking. It is defending the compound list. An examiner who reads “we docked 50 phytochemicals of Withania somnifera” will ask where the 50 came from, why those and not the other compounds in the plant, and how a plant name on a page became a 3D molecule with the right charges. Most dissertations answer that badly. This guide from the StemSkills Lab team (10+ years in structural bioinformatics, drug design and molecular modeling) walks the full path from plant name to docked pose, with every database count and command flag checked against primary sources on 17 September 2026. It is a spoke in our pillar guide to learning molecular docking, and the computational biology skills roadmap shows where this sits in the wider skill path.

How do you choose a protein target for a phytochemical study?

Choose the target before you touch any compound database. Working the other way round, collecting compounds first and then hunting for something to dock them into, produces the projects that fall apart at the viva.

A workable target has three properties. It has an experimental structure in the Protein Data Bank at a resolution you can justify, ideally 2.5 Angstrom or better. It has a co-crystallised ligand sitting in the site you care about, which gives you both a binding-site definition and a redocking control. And it has a stated link to the disease or activity your plant is traditionally used for, so the study has a reason to exist beyond convenience.

If the target you want has no bound ligand, you are committing to blind docking or a predicted pocket, and the honest write-up gets harder. Our guide on how to choose a protein target and ligands for molecular docking covers the selection logic in detail, and how to find a protein binding site for docking handles the case where no ligand is bound.

Where do you get a defensible list of plant compounds?

Four public resources cover almost every phytochemical project, and they are good at different things. Using one where another belongs is the usual reason a compound list looks arbitrary.

IMPPAT 2.0 is the right starting point for any study framed around Indian medicinal plants or Ayurvedic use. It is hosted by the Institute of Mathematical Sciences, Chennai, at cb.imsc.res.in/imppat. Version 2.0, released on 17 June 2022, captures 4,010 Indian medicinal plants, 17,967 phytochemicals and 1,095 therapeutic uses, built by manual curation from more than 100 books on traditional Indian medicine and over 7,000 published research articles. The important structural feature for a docking project is that associations are recorded at the level of plant part, so you can restrict a list to compounds actually reported in the root or the leaf rather than everything ever found anywhere in the species.

Dr. Duke’s Phytochemical and Ethnobotanical Databases, maintained by the USDA at phytochem.nal.usda.gov, is the ethnobotanical counterpart. It links plants, chemicals, biological activities and traditional syndromes, and the interface currently indexes 49,788 records across those entity types. It is strongest when your research question starts from a reported activity rather than a species. The site notes that search results can be downloaded in spreadsheet form, which makes it practical for assembling a starting list. It does not give you chemical structures, so every hit still has to be resolved elsewhere.

COCONUT (COlleCtion of Open Natural prodUcTs), at coconut.naturalproducts.net, is the broad open natural-product space rather than an ethnobotanical resource. It aggregates openly available natural-product datasets with names, synonyms, source organisms, geographic collection information and literature references. COCONUT 2.0 rebuilt the web application and re-curated the data. The paper describing it records that the dataset is available for bulk download as SDF, CSV or a database dump, with structure, substructure and similarity searching on the site. Use it when you need chemical breadth or a similarity search around a known active.

NPASS at bidd.group/NPASS is the one to reach for when you want experimental activity attached to natural products. Version 3.0, released on 15 June 2025, reports 204,023 natural products from 48,940 source organisms, mapped to 8,764 biological targets across 1,048,756 activity records. That last column is what the others lack. If you can show that a compound in your list already has measured activity against a related target, your selection rule stops looking arbitrary.

PubChem is not a phytochemical database, and that is exactly why it belongs in the workflow. It is the identifier hub that turns four incompatible compound lists into one reproducible set with 3D structures.

Which phytochemical database should you use?

A practical inclusion rule reads something like this: every phytochemical reported in IMPPAT 2.0 for the root of the species, restricted to those that resolve to a single PubChem CID and pass a stated drug-likeness filter. Two sentences, reproducible by anyone, and it survives questioning. That is the whole point.

How do you turn a compound name into a PubChem CID?

Compound lists arrive as names, and names are not identifiers. The same molecule appears as different synonyms across sources, and the same common name sometimes covers several stereoisomers. Resolve everything to a CID before doing anything else.

PubChem’s PUG-REST interface does this from the command line. To get the CID for a single compound:

curl "https://pubchem.ncbi.nlm.nih.gov/rest/pug/compound/name/quercetin/cids/TXT"

That returns 5280343. The same call for withaferin A returns 265237, and curcumin returns 969516. Loop it over your list and keep the name-to-CID mapping as a file, because that mapping is the reproducibility record your methods section needs.

Once you have CIDs, pull the 3D conformer directly:

curl "https://pubchem.ncbi.nlm.nih.gov/rest/pug/compound/cid/5280343/SDF?record_type=3d" -o 5280343.sdf

And pull the properties you will filter on in one request:

curl "https://pubchem.ncbi.nlm.nih.gov/rest/pug/compound/cid/5280343/property/MolecularWeight,XLogP,HBondDonorCount,HBondAcceptorCount,RotatableBondCount/CSV"

For quercetin that returns a molecular weight of 302.23, XLogP of 1.5, five hydrogen-bond donors, seven acceptors and one rotatable bond. Those five numbers are enough to apply a first-pass filter across a few hundred compounds in seconds.

Should you filter the compound list before docking or after?

Before. Filtering after docking is how students end up with a results table of 200 scores that nobody, including them, can interpret.

The argument for filtering first is made well by the IMPPAT 2.0 authors themselves. Writing in ACS Omega, Vivek-Ananth, Mohanraj, Sahoo and Samal report filtering the full library down to “a subset of 1335 drug-like phytochemicals”, most of which have no similarity to existing approved drugs (IMPPAT 2.0, ACS Omega 2023, 8(9), 8827-8845). Roughly 1,335 of 17,967 compounds clearing a drug-likeness filter tells you what the raw phytochemical space looks like: mostly sugars, fats, pigments and structural polymers that were never plausible ligands.

Apply the filter at the property level using the PubChem values you already pulled, then run the survivors through SwissADME or pkCSM for a proper absorption and toxicity read. Our post on drug-likeness and ADMET with SwissADME and pkCSM covers the parameters and how to report them. Docking tens of well-chosen compounds and discussing each one properly beats docking thousands and discussing none.

State your cut-offs explicitly in the methods. “Compounds with molecular weight above 600 or more than 10 rotatable bonds were excluded” is a defensible sentence. “Drug-like compounds were selected” is not.

How do you prepare phytochemical ligands for AutoDock Vina?

This is where these projects actually fail, and the failures are silent. A flat ligand still docks. A wrongly protonated one still returns a score. Nothing warns you.

Open Babel handles the whole chain. Release 3.2.1 is the current version, published on 11 July 2026. Work through it in four deliberate steps rather than one command.

Step 1: strip salts and counter-ions. Plant compound records frequently carry chlorides, sodium or associated solvent. The -r flag keeps only the largest contiguous fragment:

obabel ligands.sdf -O clean.sdf -r

Step 2: set protonation for physiological pH. Phenols, carboxylic acids and amines are everywhere in phytochemistry, and their charge state at pH 7.4 changes the hydrogen-bond pattern completely. The Open Babel command-line documentation defines -p <pH> as “Add hydrogens appropriate for pH”. Apply it:

obabel clean.sdf -O protonated.sdf -p 7.4

Read the caveat in the same documentation before you trust the output. Protonation is applied atom by atom, so a molecule with several ionizable centres has every one of them ionized. For a polyphenol with several acidic hydroxyls that behaviour is often wrong, so inspect a few structures by hand rather than accepting the batch result. The same logic applies on the receptor side, which we cover in protein protonation states and adding hydrogens for docking and MD.

Step 3: generate real 3D coordinates. An SDF downloaded as 2D has every atom in a plane, and every pose you get from it will be meaningless. Open Babel will not generate coordinates unless asked. Use --gen3d with an explicit quality setting:

obabel protonated.sdf -O ligands3d.sdf --gen3d slow

The documented speed options run from fastest (no cleanup) through the default med (force field cleanup of 100 cycles plus a fast rotor search) to slowest (500 cycles plus a slow rotor search). For a few hundred flexible natural products, slow is a reasonable trade. If you download 3D SDF straight from PubChem as shown above, you can skip this step for those compounds.

Step 4: convert to PDBQT. AutoDock Vina reads PDBQT, which carries partial charges and torsion definitions:

obabel ligands3d.sdf -O ligand.pdbqt -m

The -m flag splits a multi-molecule file into one PDBQT per compound, which is what a batch run needs. Check available charge models with obabel -L charges if you want to set one explicitly with --partialcharge. Our guide to converting molecular file formats with Open Babel goes deeper on the conversion options, and preparing the protein and ligand for docking covers the receptor side.

How do you run and validate the docking itself?

AutoDock Vina 1.2.7, released on 26 February 2025, is the current release from the Scripps Research Center for Computational Structural Biology. The mechanics are the same as any Vina run, so rather than repeat them here: define the grid box from the co-crystallised ligand as described in how to set the grid box in AutoDock Vina, and script the batch as in docking multiple ligands with AutoDock Vina.

Two validation steps separate a study an examiner accepts from one they do not.

First, redock the co-crystallised ligand into its own site before you dock anything else, and report the RMSD against the crystal pose. If your protocol cannot reproduce a known answer, its scores for unknown compounds carry no weight. Second, fix and report your exhaustiveness and random seed, because Vina is stochastic and a rerun will not reproduce your table otherwise. We cover both in AutoDock Vina exhaustiveness and reproducible docking.

What do the scores actually mean for plant compounds?

A docking score is not activity. This matters more for phytochemicals than for synthetic libraries, and saying so plainly in your discussion is a mark of competence, not weakness.

Plant extracts are rich in polyphenols, quinones, tannins and catechols. Many of these are well-known assay-interference compounds that produce apparent activity across unrelated targets through aggregation, redox cycling or covalent reactivity rather than specific binding. Scoring functions reward exactly what these molecules have in abundance, which is many hydrogen-bond donors and acceptors and a large contact surface. A strong score on a promiscuous polyphenol is the least informative result in the field.

What makes a phytochemical docking study credible is not the score table. It is a compound list with a stated inclusion rule, a protocol validated by redocking, and follow-up on the top few hits rather than a ranked list of everything. That follow-up means molecular dynamics on the best complexes and a binding free energy estimate, which our post on binding free energy with GROMACS and MM-PBSA walks through. Interpreting what you have is covered in how to interpret molecular docking results, and the contacts are best shown with a protein-ligand interaction diagram using LigPlot+ or PLIP.

What goes wrong, and how do you fix it?

These are the failures that show up repeatedly in phytochemical projects, with what to do about each.

A compound name returns no CID. The PubChem REST call returns HTTP 404 for a name it cannot resolve. Try the synonyms listed in the source database, search the IUPAC name, or search by structure if the source gives you a SMILES string. If nothing resolves, drop the compound and say so in the methods rather than quietly leaving a gap.

A name returns several CIDs. Common names often cover stereoisomers. Searching catechin returns more than one CID, and picking the first silently commits you to an arbitrary stereochemistry that changes the docked pose. Check which isomer your source actually reported and select it deliberately.

Every docked pose looks flat. The input SDF was 2D. Confirm by opening the file and checking whether the third coordinate column is all zeros, then regenerate with --gen3d.

PubChem has no 3D record. Adding record_type=3d returns a 404 for compounds PubChem has not computed a conformer for, which includes many large natural products. Download the 2D record instead and build coordinates yourself with --gen3d.

The PDBQT has zero rotatable bonds. Vina then treats the ligand as rigid and the result is not a meaningful docking. This usually means the input had no 3D coordinates when it was converted. Fix the coordinates first, then reconvert.

The compound is too large or too flexible. Tannic acid (PubChem CID 16129878) has a molecular weight of 1701.2 and 31 rotatable bonds. No rigid-receptor docking score for a molecule like that means anything, and PubChem has no 3D conformer for it either. Exclude such compounds at the filtering step and state the cut-off.

Duplicate compounds across plant names. The same molecule is reported under several species and several synonyms, so a merged list double-counts. Deduplicate on CID, never on name, and report the count before and after.

Frequently asked questions

Is molecular docking of phytochemicals enough for an MSc dissertation?

Docking alone is usually treated as thin. A defensible project pairs a justified compound list with a redocking-validated protocol, then adds molecular dynamics and a binding free energy estimate on the top two or three hits. That combination is what examiners expect to see.

Can I dock a whole plant extract?

No. Docking works on single, defined molecules. What you can do is dock every compound reported for that plant part in a curated database, which is a different and more defensible claim. Say “compounds reported in the root of the species”, not “the extract”.

Do I need IMPPAT if I already have a list from a review paper?

A review paper list is acceptable if you cite it and state the inclusion rule, but it is usually incomplete and rarely plant-part specific. Cross-checking against IMPPAT 2.0 or COCONUT either confirms your list or shows you what was missing, and both outcomes strengthen the write-up.

Which software do I need, and does it cost anything?

Nothing in this workflow is paid. AutoDock Vina, Open Babel, PubChem, IMPPAT 2.0, Dr. Duke’s, COCONUT, NPASS and SwissADME are all free for academic use. Check licence terms before any commercial use: IMPPAT, for example, is released under Creative Commons Attribution-NonCommercial 4.0.

How many phytochemicals should I dock?

There is no correct number, but tens beats thousands for a student project. A focused set you can filter, dock, validate and discuss individually produces a better dissertation than a large screen you cannot interrogate. Let your stated inclusion rule set the number rather than choosing one in advance.

Why did my top hit score better than the co-crystallised ligand?

Usually because it is larger. Vina’s scoring function is not size-normalised, so bigger molecules with more contacts tend to score better regardless of whether the binding is specific. Compare ligand efficiency alongside the raw score before claiming anything.

Where to go next

The workflow in short: choose a target with a bound ligand, build the compound list from a curated source that matches your research question, resolve every entry to a PubChem CID, filter on properties before docking rather than after, prepare ligands with attention to salts, protonation and real 3D coordinates, validate the protocol by redocking, and follow up the top hits with MD rather than expanding the score table. Work through the molecular docking pillar guide for the full sequence.

Sources

• Vivek-Ananth RP, Mohanraj K, Sahoo AK, Samal A. IMPPAT 2.0: An Enhanced and Expanded Phytochemical Atlas of Indian Medicinal Plants. ACS Omega 2023, 8(9), 8827-8845.
• Chandrasekhar V, Rajan K, Kanakam SRS, Sharma N, Weissenborn V, Schaub J, Steinbeck C. COCONUT 2.0: a comprehensive overhaul and curation of the collection of open natural products database. Nucleic Acids Research 2025, 53(D1), D634-D643.
• Dr. Duke’s Phytochemical and Ethnobotanical Databases, US Department of Agriculture. Checked 17 September 2026.
• NPASS: Natural Product Activity and Species Source Database, version 3.0, released 15 June 2025.
• Open Babel command-line documentation, release 3.2.1.
• AutoDock Vina, release 1.2.7, Scripps Research.