Counted and Visible Toolkit

1.   Takes a holistic approach to capacity development

   The toolkit includes strategies and mechanisms on coordination, communication, advocacy and use of disaggregated gender statistics using existing data from household surveys to ensure institutionalized and sustainable gender data production.

2.   Showcases country-led experiences

   Initiatives highlighted are those that have been directly implemented by national statistical systems to ensure the Toolkit “talks” to their counterparts in other countries with similar contexts.

3.   Utilizes existing data from household surveys

   Recognizing the constraints of conducting new household surveys, the toolkit shows how to achieve the full potential of existing household surveys and produce statistics that leave no one behind.

4.   Serves as a living document

   UN Women and the Intersecretariat Working Group on Household Surveys (ISWGHS) consider this as a dynamic document that will be periodically updated as other countries use and adapt the Counted and Visible Toolkit to their country-specific needs.

5.   Uses examples from Women Count countries

   Since the list is not exhaustive, we welcome suggestions for country examples that may be added

1.   Prescribe gender data production processes

   While this toolkit serves as a guidance, it is not intended as the final word on how existing data from household surveys can be better utilized to produce disaggregated gender statistics. Further, it does not require all stages be undertaken by users and in a sequential manner, and the stages outlined are not exclusive to the process of producing disaggregated gender statistics using existing data from household surveys.

2.   Provide recommendations towards rationalizing existing household survey systems

   While the toolkit focuses on extracting more value from the existing household survey system, its rationalization, coordination, and harmonization is beyond the scope of this Toolkit and more aligned with the overarching work of the ISWGHS.

Engaging stakeholders is key to generating commitment and ensuring that disaggregated gender statistics are produced, used, and institutionalized into a gender-responsive NSS. Here are a few points to keep in mind:

•   Shared leadership between the NSO and national women’s machinery

  Ensuring co-ownership and engagement of key actors in the NSS – particularly the major gender data producers and users – is critical in the production and use of disaggregated gender statistics. The process needs to be user-oriented.

•   A participatory and coordinated approach

  The system for gender statistics is highly decentralized, in that gender data is collected and produced not only the NSO but also by sectoral ministries and other major data producers. It thus requires engaging a broad range of core political and public actors. So, NSS-wide consultations should involve different sectors and local governments as well as users and data producers of gender statistics beyond government institutions.

Active coordination is required within the NSS to get the commitment, engagement, and participation of gender data users and stakeholders. Here are some ideas to improve coordination from the Toolkit:

• Designation of gender statistics focal points or establishment of a gender statistics unit within the NSO or other key actor of the NSS
• Formation of an inter-agency group or task force on gender statistics within the NSS
• Legislation, statistical policies and agreements on the utilization of existing household surveys to generate disaggregated gender statistics
• Development of a multi-year work program for the generation of disaggregated gender statistics
• Web portal for gender statistics

As a major step for engendering the statistics process in the country, the Chief Statistician of Cameroon pushed for the creation of a gender statistics unit within the National Institute of Statistics (NIS) – a clear demonstration of the NIS leadership’s commitment of this initiative. This led to the creation of a Permanent Working Group on Gender Statistics in 2019, led by the Chief Statistician and serving as the gender statistics think thank within the NIS. It is tasked to coordinate gender-sensitive statistical activities within the NIS as well as the broader NSS. Key achievements from this working group includes the identification and adoption of a minimum set of priority gender indicators and the production of policy briefs.

UN Women supported a statistical legislation that was passed in 2020 to regulate the statistical activities in the country, updating the law in place since 1991. As gender statistics contribute to the country’s official statistics, the legislation implicitly provides a basis for data collection needed for disaggregated gender statistics. The law also calls for the national government to provide the necessary budget for the statistical operations of the country.

Led by the National Statistics Office of Georgia (GEOSTAT), the country has been building a gender-responsive statistical system through a National Strategy on Gender Statistics and establishing the Inter-Agency Working Group on Gender Data (IAWG-GD). The Working Group serves as the country’s mechanism for cooperation and coordination of gender statistics. It also helped identify the National Priority Gender Equality Indicators (NPGEIs), which will inform the level of disaggregation of gender statistics that will be compiled on a regular basis using existing data from household surveys and other data sources. Importantly, the Working group contributed to the assessment and development of new policies, particularly the National Action Plans on combating violence against women and domestic violence, human rights, and Security Council Resolutions on women, peace, and security.

• Framework and Implementation Guidelines for Assessing Data and Statistical Capacity Gaps for Better Gender Statistics [PARIS21]  
• Integrating a Gender Perspective into Statistics [UN Statistics Division]  
• Making data count for all: Good practices in integrating gender in national statistical systems [UN ESCAP, UN ECE, and UN ESCWA]  

• UN Women Asia and the Pacific. Training Syllabus: Curriculum on Gender Statistics Training. Module 4: User-Producer Dialogue. Women Count Programme. Bangkok, Thailand. December 2020.  

Viet Nam’s NPGEIs were endorsed in 2019. During its development it followed the guidelines of the EPIC tool developed by ESCAP, which includes two stages and the following steps:

Stage I: Preparing for the analysis

• Step 1: Identify EPIC team members or potential members to form a team for analysis
• Step 2: Identify a policy document for the analysis
• Step 3: Identifying sections of policy documents
• Step 4: Read and familiarize definition of issues for action (IA), target groups (TGs), core concepts (CCs)
• Step 5: Identify/prepare frameworks for review: national, regional and global indicator sets

Stage II: Carrying out the analysis

• Phase 1 (Step 6-7-8): Identify of IA and TGs and establishing linkages across various dimensions of development through CCs
• Phase 2 (Step 9-10-11): Towards the development of national indicators set
• Phase 3 (Step 12): Identification of additional IA and TGs for future planning and consideration

In estimating gender statistics with multiple levels of disaggregation, the estimates in a ‘smaller’ domain or subdomain need to be identified. Subdomains can be geographical or formed by a cross-classification of certain population attributes. Generally, subdomains are identified depending on the purpose of estimation like an indicator or a policy to which the disaggregated gender equality statistics will be used.

National household surveys are designed with specific domains of estimation, which have a sufficient number of observations to produce relatively reliable estimates. In producing disaggregated gender statistics, the estimates will be obtained from ‘smaller’ domains or subdomains.

In direct estimation, only the collected or observed data in the subdomain are used in the computation. In other words, the direct estimator is a weighted total of the observations from all sampling units in the subdomain collected or obtained through a nation-wide survey. The survey sampling weight is the reciprocal of the probability of inclusion of an observational unit in a nation-wide survey and is computed based on the sampling design of the survey.

Note: Sampling weights should be included in the estimation process as the estimates should reflect the achieved weighted values in accordance to the sample survey sampling procedure used in conducting the national survey. The incorporation of the sampling weights ensures the accuracy of the estimates calculated. However, the estimator used does not guarantee that the estimates are precise as this can only be established using the observations gathered for the variable of interest.

NSO Mongolia and UN Women jointly analyzed the indicator, ‘Proportion of people who did not complete more than six years of education (or those who are education-poor)’ using Mongolia’s Multiple Indicator Cluster Survey (MICS) 2014–15. Computations show that the likelihood of being education-poor increases if women and girls identify with ethnic minorities, religious minorities and live in a poor household. These factors compound to create substantially deprived groups of women. NSO Mongolia also used its MICS 2018 to estimate the indicator, ‘Proportion of women who married as children’. Undertaking a gender and intersectionality analysis, smaller subdomains were also considered in estimating the indicator, such as the proportion of child marriage among women aged at least 18 years residing in urban areas by wealth index quintile.

See annex 5 in the Toolkit’s executive summary. 

The case of Pakistan shows how Geographic Information System (GIS) data can be used to highlight the relationship between gender-based deprivations and location-based inequality from the intersection of geography with other group-based inequalities. The GIS modules of Pakistan’s Demographic and Health Survey (DHS) 2017 were used to capturing spatially segregated socio-economic disadvantage of women and girls. The study indicates that overlapping inequalities, for example, those based on gender, ethnicity, geography, and wealth, can and often do produce a form of disadvantage that is acute and distinct, leaving women and girls facing these overlapping forms of discrimination worse off than other groups in society.

See annex 2 in the Toolkit’s executive summary. 

In 2019, UN Women implemented a programme on ‘Strengthening the Resilience of Syrian Women and Girls and Host Communities in Iraq, Jordan and Turkey’, mainly funded by the EU Regional Trust Fund for the Syria Response. Building on this experience, UN Women collaborated with FAO to produce a gender-sensitive resilience index based on FAO’s Resilience Measurement Analysis (RIMA) Model.

This gender-sensitive resilience capacity index (RCI) measures changes in programme beneficiaries’ resilience and whether this is the same for all women across communities of origin (that is, host communities, refugees and internally displaced people (IDPs)). These were generated through repeated surveys with the same group of women at different points in time.

For the resilience profile of programme beneficiaries in Iraq, as of May 2020, see annex 1 in the Toolkit’s executive summary. 

A quantitative assessment involves evaluating the statistical properties of the estimates. The Counted and Visible Toolkit focuses on the following properties:

•   Sufficiently accurate, as measured by the bias

  Accuracy refers to the closeness of estimates to the true value of the indicator. A large bias may be due to sampling error, non-sampling-error, or both. Non-sampling errors cover all types of errors from all sources such as response errors, coverage errors, and errors linked to data collection and processing. See the annex.

•   Sufficiently precise, as measured by the standard error (SE)

  Precision is a measure of closeness of the estimates to each other. SE is computed by dividing the square root of the ratio of the variance of the estimates and the number of observations used in the estimation. This can be automatically generated in statistical analysis software like STATA. In the generation of disaggregated gender statistics, the estimates are expected to be less precise as more disaggregation is applied in the data since the sample size was computed for large domain of estimation, like national or regional level. [See statistical formula and practical application using STATA.]

•   Sufficiently reliable, as measured by the coefficient of variation (CV)

  Reliability of the estimates measure variability of the value of the estimates through the CV. Mathematically, it is expressed in percent and computed as ratio of the standard error of the estimate and value of the estimate. Thus, the smaller the value of the CV, the better. There are no internationally agreed standards or recommendations as to the “acceptable” values of CV for a certain type of estimator. [See statistical formula and practical application using STATA.]

A qualitative assessment is more of a practical evaluation of the estimates to assess how the estimates are perceived, mainly by users and stakeholders of the estimates generated, to provide the true picture of reality.

While quantitative assessments are usually done by data producers, qualitative assessment usually involves prospective users and subject experts. For example, an assessment of disaggregated survey results on local level estimates of poverty among women calls for the involvement of local officials (e.g., planning and development officers, county, or village leaders) with knowledge of the local area. The insights of specialists will be useful both for producers by ensuring the accuracy, precision, and reliability of the estimates, and for promoting the use of estimates.

Some ways to conduct these assessments include:

• Workshops and focus group discussions involving prospective users and subject experts.
• Consultations with colleagues familiar with the sub-population group of interest, wherein concerned population are asked to assess whether the estimates under study match with their expected results.

Data producers are encouraged to present estimates along with corresponding CVs. Since cut-offs of quantitative assessments vary from country to country and in some cases, from survey to survey, here are some suggestions for when estimates do not meet defined thresholds:

1. Include caveats in the publication of results to be transparent about the reliability of the estimates. Standard errors and confidence intervals may also be presented as supplemental indicators. In practice, CV thresholds vary country to country and in some cases, from surveys to surveys. Some literature regards a measure of CV less than 10 percent as highly acceptable while a CV with value between 10 and 20 percent as still acceptable. For CV values ranging between 20 and 33 percent, estimates are regarded as less acceptable but still sufficiently reliable and should be used with caution. For those greater than 33 percent, caveats should be provided in terms of the level of reliability of these estimates.
2. Provide information on the sampling design and explanations on why certain estimates do not meet criteria as it can help planning of future surveys and studies.

Disaggregated gender statistics were generated for nine gender indicators using Mongolia’s MICS 2018 for child marriage estimates using the wealth index and type of residence as disaggregating variables: one indicator used only one disaggregation variable; four indicators used two variables; and the remaining four used three variables. Examining the reliability of the estimates as measured by their corresponding CVs, only five of eight estimates are relatively reliable. Two estimates with CVs greater than 10 but less than 20 percent are relatively sufficiently reliable but should be used with caution. On the other hand, a caveat should be provided when publishing the remaining indicator given a CV of 30 percent.

See Annex 5 of the toolkit executive summary.  

The then Philippines National Statistical Coordination Board (NSCB; now part of the Philippine Statistics Authority (PSA))4 employed small area estimation (SAE) techniques based on the Elbers, Lanjouw and Lanjouw (ELL) methodology developed by the World Bank. Almost two decades ago, this generated – for the first time – intercensal small area estimates of poverty of all 1,622 cities and municipalities (as of 2000). Given the nature of drilling down to very low levels of geographic disaggregation, the SAE exercise called for the examination of the reliability of these small area estimates. Coefficients of variation as well as standard errors and confidence intervals were made available to all users, published along with the release of the small area estimates. This practice was institutionalized in all succeeding SAE exercises undertaken by the PSA. Further, this good practice of integrating and institutionalizing publication of measures of precision and reliability of estimates were also applied in the release of official estimates of poverty, directly generated from results of the Family Income and Expenditures Survey (FIES).

See Annex 4 of the toolkit executive summary. 

• Multilevel disaggregation analysis to monitor the SDGs from a Leave No One Behind perspective [UN Women]. 
• Introduction to Small Area Estimation Techniques: A Practical Guide for National Statistics Offices [ADB]. 
• Compilation of tools and resources for data disaggregation [IAEG-SDGs and UNSD]. 
• Practical Guidebook on Data Disaggregation for the SDGs [ADB]. 
• STATA Manual. Proportion Command. 
• STATA Manual. Svy Command. 

• Disaggregation of the SDG indicators related to food and agriculture [FAO]. 
• User Guide for the Survey of Household Spending 2017. Household Expenditures Research Paper Series [Statistics Canada]. 
• Healthy People 2010 Criteria for Data Suppression [CDC, Healthy People 2010] 

• 2015 Family Income and Expenditure Survey. Quezon City, Philippines [PSA]. 
• 2012 Municipal and City Level Poverty Estimates [PSA]. 

A communication and advocacy plan should be developed, highlighting the importance of using gender statistics to inform gender-responsive policymaking. Having a well-developed plan signals the move from traditional dissemination towards more holistic communication and advocacy. Importantly, it should also reflect deeper engagement on gender and intersecting inequalities to better address them in statistical and policy work.

The plan should outline key components of dissemination and communication such as:

•   Communication and advocacy team.

  The team should include, among others: specialists on statistics, gender policy, and communication, who can serve as messengers.

•   Target users.

  Gender data users vary in terms of the types of gender data they need, their statistical literacy, and their interest and use of gender statistics. Communication tactics should recognize these distinctions and be tailored to these different users.

•   Key messages

  Messages should be driven by users’ needs and demands.

•   Tools

  There is a lot of guidance on tools and media that can help convey the messages, such as data stories, infographics, social media objects, and data visualization – see the training modules below, from UN Women and SPC in particular

•   Timeline and frequency

  Identifying the timeline of a release will aid in planning both in the processing of the data as well as outreach efforts leading up to and beyond the release.

• Communication and advocacy activities. These are covered in more detail below.

Communication and advocacy are essential to ensure that gender statistics not only reach their target audience but are used to drive policy change to advance gender equality. This includes:

•   Promotion of statistical products and events.

  This can be done via various channels, including the NSS website, social media assets, and e-mail distribution lists. Relationships with media outlets should be nurtured to encourage them to cover new data releases and incorporate the new data in their stories. PARIS 21 and Women Count designed an e-learning course to identify points of cooperation and collaboration between journalists and statisticians.

•   Advocating for the use of gender statistics.

  It is essential to maintain good relationships with users, and encourage their use of gender statistics through activities such as training courses for statistics literacy, generating user-friendly data visualizations, data fact sheets and creating opportunities to build dialogue between users and producers of data.

•   Monitoring use.

  A monitoring plan should systematically get updated on quantitative use data such as downloads of statistical products, attendance at events, and use of gender statistics. Qualitative use cases should also be pursued to track the depth of usage. The Women Count programme developed a guidance to systematically monitor the use of new gender data that came out of nearly 50 rapid gender assessments (RGAs) on the impact of COVID-19

In November 2020, Colombia released its first edition of Women and Men: Gender Gaps in Colombia. The launch was opened by the Vice President of Colombia, Marta Lucía Ramírez, demonstrating the highest-level government support, and received wide attention online. Less than a month since the launch of the report, Colombia’s NSO committed to institutionalizing the report as an annual publication, and to producing regional reports to inform local decision-and policy-making.

The success of this publication highlights the role of partnerships in communicating gender statistics:

• Partnerships with other government stakeholders. The publication was jointly produced and launched by the NSO, the Presidential Advisory Office on Gender Equality (CPEM), and UN Women Colombia. High-level government commitment ensured the wide uptake of the publication.
• Partnership with media. The launch was organized in partnership with El Espectador, a national media outlet in Colombia with 4 million Facebook followers. The event was covered in at least 47 publications across the country.
• Partnership with UN Women Colombia. The UN Women Country Representative in Colombia is a staunch advocate of communicating gender statistics, and strongly supported the launch activities.

In August 2020, Kenya launched its first ever Women’s Empowerment Index (WEI). UN Women, through the Women Count programme and in partnership with Kenya National Bureau of Statistics (KNBS) and UNICEF, co-led the production and launch of Kenya’s WEI. WEI is a major milestone in the country’s monitoring of SDG 5 as the first comprehensive and systematic measure for women’s and girls’ empowerment in Kenya. The WEI launch received significant mainstream media coverage.

The success of the WEI uptake was the result of targeted communication to a broad audience:

• Dedicated communications capacity. A Communications Consultant recruited by the Women Count programme in East and Southern Africa, provided expertise in developing key messages and packaging the publications finding to target broader audiences. This tailoring of the product made it more accessible for media coverage.
• Companion assets – Women Count developed a set of infographics as a companion to the report to help visualize and summarize complex data. These infographics were used across social media and resulted in a wider uptake.

In updating its NPGEIs in 2019, the Uganda Bureau of Statistics (UBOS), in collaboration with other Ministries, Departments and Agencies (MDAs) embarked on reprocessing existing census, survey and administrative data to provide disaggregated statistics on select NPGEIs, particularly those related to SDG Tier 1 indicators. As a result, reporting of gender indicators in Uganda’s Voluntary National Reviews (VNRs) have increased by 150% in a span of four years: from 11 in 2016 to 28 in 2020.

The increased usage of the NPGEIs in national reporting are due to improvements in disseminatation:

• Dissemination plan. Uganda developed its own dissemination plan for gender statistics, which included an expanded social media presence, increasing awareness of UBOS data.
• Creation of URL shortcuts to facilitate access. One of the simplest tactics was the use of a URL shortener to provide users with simple and accessible links to NPGEI publications,.
• Gender Statistics Portal. The UBOS Gender Statistics Portal is a one-stop center for gender statistics in the country, providing access to gender statistics under different themes, along with news and upcoming events.

• Training module on Communicating Gender Statistics (Module 10) [UN Women] 
• Guidance on Communicating gender Statistics [UNECE] 
• Guide to gender statistics and their presentation [The Pacific Community (SPC)] 
• A Three-Part Guide Series on Making Data Meaningful [UNECE] 
• Communicating with the Media: A guide for statistical organizations [UNECE] 
• Best Practices in Designing Websites for Dissemination of Statistics [UNECE] 
• Dissemination of Microdata Files: Principles, Procedures, and Practices [IHSN] 

• E-Learning Course on Communicating Gender Statistics [PARIS21 and UN Women] 
• Regional Workshop on Communicating gender data promoting better use and delivering impactful messages [UN Women] 
• Every child counts: Using gender data to drive results for children [UNICEF]