ECE250 Testcases

This repository contains a collection of MIT-licensed testcases for each project in ECE250.

Project	Number Of Testcases Available
Project 1	23 (22 active in manifest)
Project 2	34 (34 active in manifest)
Project 3	31 (30 active in manifest)
Project 4	21 (21 active in manifest) + 12 datasets

Sweet, but how do I use them?

Follow through the instructions in the "Setup" section, then actually test your project by following the "Automatic Testing" section.

Setup

As recommended time and time again by the teaching staff, it is highly recommended you do your testing from an eceubuntu server. Steps to connect to such a server are beyond the scope of this README.

Once you're logged into an eceubuntu server (or a local Unix/Linux system of your own), clone this repository:

$ git clone https://github.com/JZJisawesome/ece250-testcases
Cloning into 'ece250-testcases'...
remote: Enumerating objects: 738, done.
remote: Counting objects: 100% (302/302), done.
remote: Compressing objects: 100% (217/217), done.
remote: Total 738 (delta 134), reused 208 (delta 78), pack-reused 436
Receiving objects: 100% (738/738), 644.47 KiB | 3.08 MiB/s, done.
Resolving deltas: 100% (366/366), done.

If you already have a checkout but want to get the latest testcases and AutoGrader fixes, you can do this instead (from within your previous clone):

$ git pull
remote: Enumerating objects: 246, done.
remote: Counting objects: 100% (245/245), done.
remote: Compressing objects: 100% (164/164), done.
remote: Total 235 (delta 116), reused 169 (delta 66), pack-reused 0
Receiving objects: 100% (235/235), 99.48 KiB | 1.36 MiB/s, done.
Resolving deltas: 100% (116/116), completed with 5 local objects.
From https://github.com/JZJisawesome/ece250-testcases
   8820b46..0e35dfb  main       -> origin/main
[...]

Automatic Testing

The autograder implementation included in this repo, called the Jekel AutoGrader, must be run from the repo root directory. Note down the path to your submission tarball, and open a terminal in the root of the repo. You can then test your tarball with

$ python3 ./jekelautograder.py path/to/your/tarball/example_p1.tar.gz
     _      _        _      _         _         ____               _
    | | ___| | _____| |    / \  _   _| |_ ___  / ___|_ __ __ _  __| | ___ _ __
 _  | |/ _ \ |/ / _ \ |   / _ \| | | | __/ _ \| |  _| '__/ _` |/ _` |/ _ \ '__|
| |_| |  __/   <  __/ |  / ___ \ |_| | || (_) | |_| | | | (_| | (_| |  __/ |
 \___/ \___|_|\_\___|_| /_/   \_\__,_|\__\___/ \____|_|  \__,_|\__,_|\___|_|    for ECE 250
Copyright (c) 2023 John Jekel, Aiden Fox Ivey, and ECE 250 Teaching Staff

Performing some basic sanity checks before we get started...
Looking good, I think I'm set to go!

To begin, let's check your tarball's file name and path...
Excellent! Based on the name of the tarball, example_p1.tar.gz, I've deduced the following:
Your UWID is: example
Your tarball is for: Project 1
[...]

The autograder will then test your tarball not only on the testcases for the correct project, but it will also check some other things:

• That the tarball is named correctly (it must be in order for the script to successfully infer the project version)
• That your design document is present and named correctly (matches the uwid in your tarball's name, etc.)
• That your Makefile is present and successfully produces an a.out file
• That you didn't accidentally include a precompiled a.out file in your tarball
• That your tarball dosn't contain any directories

Just like the real thing, the autograder will use Valgrind, so it is quite good at catching mistakes!

(It uses Leaks on MacOS)

Manual Testing

This is useful if you are debugging an issue with your code that is triggered by a particular test case

The projects/ folder contains several subdirectories, one for each project. Within the project folder you're interested in, there is an "input" and an "output" folder.

Each file in "input" is a testcase you can run with the following command (assuming you have Valgrind installed and your code is compiled to the binary a.out)

$ valgrind ./a.out < path/to/testcase.in
[...]

You can then compare the results to the corresponding file in the "output" folder to ensure your program produces the correct output!

TODO instructions for manual testing with Leaks

Testcase Descriptions

Project 3

Testcase	Description
alphabetic_print	Loads the corpus and prints it. A good way to check your are printing correctly in alphabetical order.
can_you_count	Testcase to test the count prefix command.
chatgpt_autogenerated_{0, 1, 2}	These are testcases autogenerated by ChatGPT, with some manual fixes. They can be hit or miss with regards to their utility in debugging/testing.
dont_erase_too_much	Erases words and ensures you don't accidentally erase other ones by mistake. Also makes sure your size variable remains consistent with the number of words present.
double_erase	Erases a word twice and ensures the behaviour is as expected
erasing_clear	Loads the corpus and clears it, then prints, inserts and removes to the empty trie.
illegal_erase	Attempt to erase a word with both valid and invalid characters to ensure the valid characters stay unerased
illegal_insert	Input a word with both valid and invalid characters to ensure the valid letters are not inserted
illegal_prefix	Input a prefix with both valid and invalid characters to ensure prefixes are counted properly and error messages are prioritized
insanity	JZJ's classic insanity test. This is the biggest one yet (almost 1 MB), but it shouldn't take that long since the trie is pretty efficient as data structures go!
im_empty_inside	Tests to ensure commands work properly when the trie is empty.
invalid	Tests how you handle invalid input for i, c, and e.
LEARN_test0{1, 2, 3, 4}	Tests provided by the ECE 250 Teaching Staff (from LEARN).
long_words	Tests words that are much longer than normal.
more_spellchecking	A few additional spellcheck commands I sort of accidentally came up with while trying to create spellcheck_no_suggestions.
multiload	Performs multiple loads and clears, using empty and size to ensure everything worked
nested_erase	Inserts a word and another word embedded in the previous word and erases both words.
prefix_example_from_spec	Tests the example situation mentioned in the spec on LEARN, with CAR, CARD, and CARMEN.
prefix_peril	Tries the "c" command in a bunch of situations, trying to invoke edgecases (ex. no prefixes available, the prefix is a valid word, the prefix is near the root, etc).
sanity	JZJ's classic sanity test. Just a single line: "exit"
single_letters	A testcase that inserts, removes, and prints several words that are only 1 letter long.
spellcheck_correct	Loads the corpus, and spellchecks every word, which should print "correct" each time.
spellcheck_example_from_spec	Tests the example situation mentioned in the spec on LEARN, with YOU, YOUN, YOUNG, and YOUR.
spellcheck_no_suggestions	Loads the corpus, and spellchecks several wrong words that have no suggestions. This is an edge case: newlines should be printed, not all of the words.
spellcheck_incorrect	Loads the corpus, and spellchecks several wrong words, trying to cover several edge cases when searching for suggestions.
YodaErase	A small testcase that ensure things still work, even with entirely unique words.

Project 4

Testcase	Description
delete_missing	Ensures your code prints "failure" when you try to print vertices adjacent to a vertex that does not exist.
dkeen_45000Vertices_100000edges.in	A test to put your code through its paces with 45000 verticies and 100000 edges! Also checks that COST works after deleting and inserting a vertex.
double_insert	Ensures your code prints "failure" when you try to insert an edge twice; and that the original weight it kept if you attempt to do so.
fm_01	Loads the graph example from Chapter 23 of CLRS. A standard check for the MST and cost calculations.
fm_02	Ensure the cost function actually calls MST in order to calculate the cost each time (it dosn't just cache the value from the last MST call).
fm_03	Ensure the MST is updated correctly when a connected graph has a vertex deleted such that it becomes disconnected, and then the relevant edge is re-added with a different weight.
fm_04	A large testcase by Farzan
heavy_graph	Sort of an extension to integer_limits/vertex_weight_limits. Ensures your MST and COST don't break down if the cost of the MST is the signed integer maximum.
im_empty_inside_again	Tests to ensure your graph behaves well when it is empty
integer_limits	Tests to ensure your code can handle input near the signed integer limits
invalid	Tests to ensure your code correctly rejects invalid vertex numbers and weights
LEARN_{smallGraph, testFull}	Tests provided by the ECE 250 Teaching Staff (from LEARN). Note LEARN_testFull uses the LEARN_bigGraph dataset.
nc_01	TODO description
nc_02	TODO description
nc_03	TODO description
print_missing	Ensures your code prints "failure" when you try to print vertices adjacent to a vertex that does not exist
sanity	JZJ's classic sanity test. Just a single line: "END"
tc_01	A testcase to test small graphs.
update_adjacent_adjacency_lists	Ensure that, when you remove a vertex, the adjacency lists of vertices that were adjacent to the vertex are updated too.
vertex_weight_limits	Tests to ensure your code works correctly even when vertex numbers and weights are near their minimums and maximums.

Dataset	Description
dkeen_45000Vertices_100000edges_dataset.in	A dataset with 45000 verticies and 100000 edges!
empty.in	The dataset with no edges in it
fm_CLRS_example.in	The graph example from Chapter 23 of CLRS, used by the fm_01 testcase.
fm_dataset_01.in	Dataset for use with the fm_02 testcase
fm_dataset_02.in	Dataset used by fm_03.
fm_fairly_massive_dataset.in	A large dataset used by fm_04
LEARN_bigGraph.in	Dataset provided by the ECE 250 Teaching Staff (from LEARN).
jzj_standard_{big, insanity, medium, normal, tiny}.in	JZJ's standard datasets (of various sizes) used by his testcases (and free for others to use for their testcases too!)

Contributing

By having your testcases included in this repository, you agree to having them released under the MIT License (of course you retain copyright). You'll get a spot in the following "Contributors" section of the README for your contributions, indicating which test files are yours.

Only input files with a corresponding expected output file will be accepted for inclusion. Furthermore, as of Project 3, you must also provide a description of your testcase (and datasets) in this README.

Just submit a PR to participate, or if you're a big enough contributor, I'll grant you maintainer status :)

Contributors

Jekel AutoGrader

Contributor
Aiden Fox Ivey
ECE 250 Teaching Staff
John Jekel

Project 1

Contributor	Number Of Testcases Contributed	Testcase Names
AlexanderTsarapkine	1	edge_cases_mem_leak
Azizul Chowdhury	1	test_REM
ChatGPT (with manual fixes)	3	chatgpt_autogenerated_0, chatgpt_autogenerated_1, chatgpt_autogenerated_2
Chris (HyperFire12#3764)	1	specificboy
ECE 250 Teaching Staff	3	LEARN_test01, LEARN_test02, LEARN_test03
Farzan Mirshekari	3	ECE250-Projects-Testcases_test04, ECE250-Projects-Testcases_test05, ECE250-Projects-Testcases_test06
Hongjun Yun	1	hongjun_total_testing
John Jekel (JZJ)	6	insanity, list_size_1, pushpop_yum, sanity, supports_double, support_valid_cpp_names
Luc Edes	1	delete
Nathan Cheng	1	edge_cases
Nick Chan	1	doingstuff
Tri Dao	1	test_DEF

Project 2

Contributor	Number Of Testcases Contributed	Testcase Names
AndyGolow	9	ag_testcase_01, ag_testcase_02, ag_testcase_03, ag_testcase_04, ag_testcase_05, ag_testcase_06, ag_testcase_07, ag_testcase_09, ag_testcase_10
ChatGPT (with manual fixes)	3	chatgpt_autogenerated_0, chatgpt_autogenerated_1, chatgpt_autogenerated_2
ECE 250 Teaching Staff	4	LEARN_test01open, LEARN_test01ordered, LEARN_test02ordered, LEARN_test03ordered
Farzan Mirshekari	2	ECE250-Projects-Testcases_test02open, ECE250-Projects-Testcases_test03open
John Jekel (JZJ)	13	insanity_open, insanity_ordered, integer_limits_open, integer_limits_ordered, oob_open, oob_ordered, printer, respect_capacity_open, respect_capacity_ordered, respect_capacity1_open, respect_capacity1_ordered, sanity_open, sanity_ordered
Nick Chan	1	nc_open01
Reezan Visram	1	ECE250-Projects-Testcases_test04ordered
Ryan (RyEggGit)	1	linkingtest
Mihir	1	integer_limits_open_for_integers, integer_limits_ordered_for_integers

Project 3

Contributor	Number Of Testcases Contributed	Testcase Names
Adam Goldman	1	single_letters
Brandon Vo	3	illegal_erase, illegal_insert, illegal_prefix
Chimera	1	YodaErase
ChatGPT (with manual fixes)	3	chatgpt_autogenerated_0, chatgpt_autogenerated_1, chatgpt_autogenerated_2
ECE 250 Teaching Staff	4	LEARN_test01, LEARN_test01, LEARN_test02, LEARN_test03
John Jekel (JZJ)	15	alphabetic_print, dont_erase_too_much, im_empty_inside, insanity, invalid, long_words, more_spellchecking, multiload, prefix_example_from_spec, prefix_peril, sanity, spellcheck_correct, spellcheck_example_from_spec, spellcheck_no_suggestions, spellcheck_incorrect
Josh Quittner	1	erasing_clear
Mihir	2	double_erase, nested_erase
Nick Chan	1	can_you_count

Project 4

Contributor	Number Of Testcases Contributed	Testcase Names	Number of Datasets Contributed	Dataset Names
Dylan Keen	1	dkeen_45000Vertices_100000edges.in	1	dkeen_45000Vertices_100000edges_dataset.in
ECE 250 Teaching Staff	2	LEARN_smallGraph, LEARN_testFull	1	LEARN_bigGraph.in
Farzan Mirshekari	4	fm_01, fm_02, fm_03, fm_04	3	fm_CLRS_example.in, fm_dataset_01.in, fm_dataset_02.in, fm_fairly_massive_dataset.in
John Jekel (JZJ)	10	delete_missing, double_insert, heavy_graph, im_empty_inside_again, integer_limits, invalid, print_missing, sanity, update_adjacent_adjacency_lists, vertex_weight_limits	6	empty.in, jzj_standard_big.in, jzj_standard_insanity.in, jzj_standard_medium.in, jzj_standard_normal.in, jzj_standard_tiny.in
Nick Chan	3	nc_01, nc_02, nc_03	0	N/A
Sushant Gangwani	1	tc_01	0	N/A